Google DeepMind releases Genie 4 world model with native audio and Gemini planning

The model can generate a world from a text prompt, let users move a character and camera with a keyboard, and accept new prompts during runtime to change the weather or add characters and objects, the report said. Genie 4 extends a single simulation from about one minute in Genie 3 to two minutes. In one demonstration, a man in a brown suit played saxophone on a platform floating on a misty lake, and users could move the camera, change the weather or add people while Gemini helped maintain scene coherence.

Ethan Mollick, a Wharton professor who tested Genie 4, used the prompt a neo-Gothic Brutalist city inhabited by robots and horses to generate a virtual city in about one minute, then added a dragon and a friendly otter NPC in real time, according to the report. An a16z investor also demonstrated a project described as 'GTA Waymo', exploring a city generated entirely by AI in real time, with native audio produced alongside the world.

Genie 4's audio generation covers environmental sound, music and character dialogue. The report cited examples including waves by the sea, wind in a forest, wing beats and rushing air while riding a dragon through a valley, and distant volcanic activity. Virtual characters can respond dynamically to user input and events instead of relying only on prewritten dialogue and behavior logic, the report said.

Google DeepMind integrated Gemini into the system for planning. Gemini Omni provides the basis for the simulated world's appearance and behavior, while Gemini 3.8 Flash acts as a real-time guiding agent that observes generated frames, breaks tasks into steps and judges whether each step succeeds. In one research experiment, Gemini 4 Argon divided a task into steps while Genie 4 simulated Apptronik's Apollo humanoid robot playing checkers. Google DeepMind research head Tejas Kulkarni said a future direction is hybrid world models that combine language models' structured reasoning with world models' sensory simulation.

Google also showed physics cases including a watermelon under impact, maple seeds falling through the air and a crowded rural train station, demonstrating collision, aerodynamics, gravity and complex crowd behavior. Characters can move more naturally and react to environmental changes, and users can guide their behavior in real time, according to the report.

Genie 4 remains limited. Simulations last about two minutes, object details can change as time passes, and scenes can become inconsistent. Repeated prompt changes or extended exploration in a large environment can cause the model to lose accurate information about earlier scenes. Complex physical interactions, spatial accuracy, audio generation and the readability of text inside scenes still need improvement, the report said, adding that audio accuracy and long-term consistency are also problems.

Genie 4 is available as a research preview to invited testers. Google plans to offer it through Project Genie to Google AI Ultra subscribers and later to early partners through the Gemini API and Google AI Studio. Google has invited testers in film, games, news, education and engineering. Filmmakers could generate explorable scenes to test camera positions, lighting, character movement and dialogue; game developers could prototype environments and basic interactions. Some photojournalists have tried turning two-dimensional historical photos into explorable dynamic scenes or simulating floods and wildfires, though AI-inferred images should not be treated as accurate historical reconstructions, the report said. The report also said Runway has released GWM Worlds 2 with real-time audio and interaction, while World Labs focuses on spatial intelligence and three-dimensional reconstruction, editing and long-term consistency; AMD announced in September that it would acquire World Labs for about $8.2 billion.