
Oneiro is a Minecraft AI agent, built entirely from scratch and powered by Gemini 3.5 Flash, that both talks with players in real time and takes real actions in the game world based on that conversation. A player can ask Oneiro to do something in natural language, and the agent acts on it directly - not just responding with text, but actually interacting with the Minecraft world through a single unified API. This is the first working piece of a much larger vision. Today's biggest gap in game NPCs is that they don't remember you - every session starts from zero. Oneiro is our starting point toward a multi-agent system with persistent social memory, where an agent remembers past interactions, builds a relationship with each player over time, and eventually perceives the world visually instead of relying only on text and commands. We're a small team building this on our own live Minecraft server, which gives us a real environment with actual players to test against, rather than a synthetic benchmark. The roadmap ahead includes a persistent memory layer, a dedicated planning agent for longer-horizon behavior, and vision-based perception via screenshots - with the long-term goal of offering this as an open, self-hostable service for other Minecraft server owners.
13 Jul 2026