Isekai Engine

Isekai Engine is a Twitch stream featuring an embodied virtual avatar (Citrine) that can do anything. We use OpenAI GPT combined with a Generative Agents style ReAct loop attached to a full Linux computer, and we render the result on the web using THREE.js with an animated VRM character in a procedurally generated virtual world (using Blockade Labs) with a perception/generation loop. The resulting render is streamed to Twitch using OBS. The purpose of the product is threefold: First, we wanted to leverage the latest generative AI models to produce a virtual TV show with a unique premise: the character is real -- she can do things in the real world with her Linux computer. Second, we want to educate the world at large about how close we are getting to AGI with generative AI models, by making the latest technology accessible in the simplest possible platform: a shared stream you can hop onto and chat with. Third, we want to explore the possibilities of monetization of generative AGI models. We think this is an increasingly important social concern as generative AI threatens to displace job markets. We believe in discovering what is possible and sharing our research so that we can prepare and develop the antibodies to the future we are rapidly accelerating into.