Google DeepMind unveils Genie 4 world model with audio and real-time editing

Google DeepMind released Genie 4, its latest world-model system, in a research preview on Tuesday. The model can generate audio that matches the visual simulation, including music, ambient sounds and interactive dialogue, and allows users to modify the scene with text prompts while the simulation runs. It supports edits such as changing weather, camera angle, or adding objects and characters, and can keep a consistent scene for up to two minutes, double the duration of its predecessor Genie 3.

The new model is tightly coupled with Google’s Gemini AI, which plans actions and checks frames as Genie creates the visuals. In a related experiment, a Gemini 4 Argon model broke tasks into steps that Genie 4 simulated in real time, illustrating a potential workflow for testing robot scenarios in virtual environments.

"What we’ll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction," said Tejas Kulkarni, research director at Google, during a briefing.

Google highlighted possible applications in gaming, interactive learning and robotics, and noted that early testers are exploring film-prototyping uses.

"The real-time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful," said Diego Rivas, a Google DeepMind engineer. He cited ideation and storyboarding as examples where rapid simulation could aid creators.

Google also referenced its Veo video-generation tool, which creates high-definition clips with synchronized audio from text, image or video prompts, and mentioned a future multimodal model, Gemini Omni, that could generate video from combined inputs. However, the team acknowledged that Genie 4 remains experimental, with challenges such as detail drift over longer runs and ensuring accurate audio and legible in-world text.