tl;dr
ego is an AI-native simulation engine and platform for AI characters where users can create and share 3D animated characters, worlds, games, and scripted interactions all using natural language. Imagine being able to prompt characters that exhibit human-like behaviour into Minecraft, The Sims, or Animal Crossing into existence.

https://www.youtube.com/watch?v=bLpG4LrLf60

Why Now?

The advent of large language models and diffusion models presents an incredible opportunity to solve three seemingly intractable problems in open-world games and simulations:

1. It is hard for humans to write code and script interactions between assets, characters, and the environment to design fun games and experiences
2. It was impossible for NPCs to display human-like behavior and have true agency in these 3D worlds
3. It is hard to create new 3D assets, worlds, and non-playable characters

We founded ego to build an AI native simulation engine to solve all three problems - and we’ve started by tackling the first two.

ego is an AI-native simulation/game engine and platform where users can create and share 3D animated characters, worlds, and scripted interactions all using natural language. Imagine being able to prompt games like GTA, The Sims, Minecraft, or Animal Crossing into existence.

In the age of Gen AI, it is finally possible to build a game engine that is also a game.

Simulations on ego

Over the course of YC, we built and launched two experimental games with our simulation engine: TownWorld (a 3D simulation of the Stanford Generative Agents paper - tweet thread) and CharacterIdol (a game where user-prompted generative agents compete to entertain American Idol-style judges…also powered by LLMs). All of the character personalities, conversations, memories, and routines/movements in these games are fully generated by AI.

Townworld:

CharacterIdol:

This is where we found the massive opportunity to meaningfully make progress toward our vision of gaining usage of our simulation engine - making it really easy for a user to make and share a game featuring agents in bounded or unbounded 3D space. We are keenly aware of the sheer amount of time users spend talking to character.ai characters, as well as the sheer amount of time young creators spend learning Lua code to script simple games and experiences in Roblox. What if we enable humans to interact with agents in immersive 3D worlds without the creators themselves needing to write complex Lua scripts or learn 3D modeling? What if summoning an agent was as simple as filling out a DnD character sheet?

Why is this a big deal?

Since time immemorial, humans have built tools to bring their internal worlds to life. The sandbox, the dollhouse, a tree branch shaped like a rifle - all examples of tools we use to bolster our imaginative capacity to dream new worlds, characters, scenarios, and stories into existence. The closest we’ve come as a species to actualizing our imagination has been through the use of game engines to create simulated open-world experiences and games. AI is about to democratize the strongest tool we’ve had to realize our imaginations, and ego will be the simulation platform and engine that will power this new possibility.

Animal Crossing, The Sims, Garry’s Mod, Skyrim, Minecraft, Roblox, and GTA - all hit games that empower players to explore, create, and roleplay in immersive 3D worlds. However, video games are hard to build and even harder to keep releasing fresh content for. Outside of the studio system, independent modders have spent countless hours extending these games’ shelf life with new experiences and characters, despite heavy technical challenges, and even outright hostility from the original game publishers.

Team

Vish led the FB AI Applied Research team and the Meta Horizon Scripting team. He learned how to program by modding video games as a teenager and also wrote screenplays for films (Acceptance).

Asks

• Sign up for our waitlist at ego.live and join our discord
• Retweet (this twitter link), reshare this (LinkedIn)
• Share the launch with friends and gamers who might be interested in trying this out :)
• Try out characteridol.ai and tell us what you think
• Follow @ego_ai_ on Twitter
• Optionally follow @peggy_wang and @simulacronist on twitter
• Link us to talented 10x game devs (Unreal engine experience): JD