In 2026 "world model" shows up in reinforcement-learning papers, in LeCun's new company, in Google's playable video worlds and in NVIDIA's robotics launches. The shared core: learn a model that predicts how an environment's state or observations evolve, usually given an action, so that an agent, a planner, a test harness or a person can use the prediction instead of the real world. But what they predict, in what space, and for what purpose differ a lot; settle those questions before comparing any numbers.
Why the term is so confusing
Primary sources define it differently. For Ha and Schmidhuber (2018) it is a generative model of a reinforcement-learning environment, which lets them "train our agent entirely inside of its own hallucinated dream generated by its world model …" (abstract) and then transfer the policy back; LeCun (2022) makes it one module of an autonomous agent that predicts future states in an abstract representation space (§3); NVIDIA Cosmos (2025-01) frames it as a "digital twin" for physical AI; World Labs (2026-06) sorts world models by output into renderers (observations), simulators (state such as geometry and physics) and planners (actions). A July 2026 perspective paper by Chen et al. says plainly in its abstract that there is no consensus on what a world model is.
The term has also become a label for companies and deals: AMI Labs announced a $1.03B round on 2026-03-10 (AMI Labs), and AMD announced on 2026-09-28 that it would acquire World Labs for about $8.2B in stock (AMD press release). When you see it, do not assume which kind is meant.
Ask 4 questions first
| Question | Common answers |
|---|---|
| Predicts what? | The next latent state and reward; the next few video frames; the embedding of a future clip; a 3D scene; or only reward, value and policy |
| In what space? | A compact latent vector; pixels, or video tokens / latents; a learned embedding; explicit 3D (Gaussian splats, meshes) |
| Conditioned on what, and on actions? | Robot actions, keyboard and mouse, ego trajectory, text events; some only control the camera, some take no action at all |
| Used for what? | Planning; training a policy in imagination; simulating and evaluating other policies; generating training data; entertainment |
One question, several kinds of answer
The whole series uses one teaching example: a robot arm is about to push a cup toward the edge of a table. Will the cup fall if it pushes 2 cm, or 5 cm? The numbers are a teaching setup, not any paper's experiment. Different world models answer in completely different forms:
| Kind | What it returns for "push 5 cm" | Who uses the answer |
|---|---|---|
| Dreamer-style latent model | The next latent state, a predicted reward, and whether the episode continues (it ends if the cup falls) | A policy trained in imagination |
| MuZero-style model | Only reward, value and policy; it need not say where the cup is | Tree search |
| Video world model | The next few frames: the cup slides, maybe tips over | A person watching, a scoring evaluator, a policy using it as training data |
| JEPA | The embedding of the future clip, then its distance to the embedding of a "goal image" | A planner such as CEM |
| Interactive video world | A person presses keys to push the cup and sees the scene respond in real time | A player, or an agent under test |
6 meanings
Common usage falls into 6 kinds; each row lists 1–2 representatives:
| Meaning | Predicts | Space | Action-conditioned? | Main use | Examples |
|---|---|---|---|---|---|
| (a) RL world model | Next latent state, reward, continuation; MuZero predicts only reward, value and policy | Compact latent | Yes | Policy learning in imagination; planning (MCTS, MPC) | DreamerV3, MuZero |
| (b) JEPA / latent predictive | Representations of masked or future parts | Learned embedding; no pixel decoder needed | Not in pretraining; yes in the action-conditioned variant | Representation learning; planning by energy minimisation | I-JEPA, V-JEPA 2-AC |
| (c) Interactive generative video | Next frame or frames | Pixels (via latent tokens or latents) | Yes (keyboard, mouse, camera, text events) | Playable worlds, agent training environments, entertainment | Genie, Genie 3 |
| (d) World foundation models for robots and driving | Future multi-camera video (some add lidar or radar), sometimes actions | Pixels or latents | Yes (ego trajectory, robot actions, text) | Synthetic data, policy evaluation, closed-loop simulation; recently the policy itself | Cosmos, GAIA-2 |
| (e) 3D / spatial | An explicit 3D scene, or view-consistent frames | 3D or posed frames | Mostly camera only | Content creation, simulation assets, real-to-sim | Marble, HY-World 2.0 |
| (f) Implicit world models inside general models | Whether an LLM or video generator internally represents world state | Hidden activations, or generated outputs | n/a | Interpretability; "world simulator" claims | Othello-GPT, Sora tech report |
The easiest confusion is between (c) and ordinary video generation. Sora and Veo are prompt-in, clip-out generators; interactive world models also take actions, generate frame by frame in real time and keep the scene consistent.
A timeline
- 2018-03 World Models: VAE plus MDN-RNN plus a linear controller; the controller is trained inside the model's "dream" and then transferred.
- 2019-11 MuZero (DeepMind): the model predicts only reward, value and policy, combined with MCTS.
- 2019-12 Dreamer: actor-critic learned on imagined latent trajectories; followed by DreamerV2 (2020-10) and DreamerV3 (2023-01).
- 2022-06 LeCun's position paper proposes JEPA and planning by energy minimisation; I-JEPA (2023-01) first applies it to images.
- 2024-02 Genie (Google DeepMind): learns latent actions from platformer videos without action labels, yielding an interactive environment.
- 2025-01 NVIDIA Cosmos: open world foundation models, tokenizers and a data curation pipeline.
- 2025-06 V-JEPA 2 / V-JEPA 2-AC (Meta FAIR): pretraining on more than one million hours of video, then action conditioning, planning zero-shot on Franka arms.
- 2025-08 Genie 3: Google DeepMind's blog describes real-time 720p, 24 fps worlds that stay consistent for minutes.
- 2025-09 Dreamer 4: trained only on offline data inside a 2B-parameter video world model, it obtains diamonds in Minecraft.
- 2026-01 to 03 Project Genie opens to paying subscribers; the Waymo World Model post-trains Genie 3 into a driving simulator; DreamZero generates video and actions together and serves directly as a robot policy; AMI Labs launches, and LeWorldModel and V-JEPA 2.1 appear.
As of October 2026
- Playable video worlds went public. On 2026-01-29 Google opened Project Genie to US Google AI Ultra subscribers, itself stating that generations last only 60 seconds and may not follow prompts or physics; Runway released a GWM Worlds 2 research preview on 2026-09-03; open models include Skywork's Matrix-Game 3.0 (2026-03) and Overworld's Waypoint (from 2026-01). Most of these models' frame rates and resolutions are self-reported.
- In robotics and driving, world models moved from making data to evaluating policies to serving as the policy, as in DreamZero from NVIDIA and collaborators in February 2026. Waymo (2026-02) published no fidelity metrics; Wayve's GAIA-4 (2026-08) gave a single number, "2.5× better reconstruction faithfulness".
- The JEPA line has new papers; the new company has no model yet. V-JEPA 2.1 (Meta FAIR, 2026-03-15), LeWorldModel (2026-03-13) and H-JEPA (2026-10-05) were released; as of 2026-10-15, AMI Labs' updates page lists no model, paper or code.
- Evaluation shifted from "does it look right" to "is it useful". Shang et al.'s WorldArena (2026-02) reports a gap between visual quality and usefulness on embodied tasks; Lu et al.'s WRBench (2026-06, 23 models) finds that models returning to a scene tend to resume it as it was instead of advancing the event that was in motion.
What has settled and what is still moving
| Fairly settled | Still moving | Signals to watch next |
|---|---|---|
| Training a policy inside a learned latent model works on standard tasks: DreamerV3 ran 150+ tasks with one configuration (authors' report, Nature 2025) | How far large video models and offline-only data can go: Dreamer 4's diamond rate is 0.7% (Google DeepMind, 2025-09) | Against the 0.7% baseline as of 2026-10: offline imagination training at higher success rates, or on tasks beyond Minecraft |
| Predicting in embedding space can support zero-shot planning on lab robot arms: V-JEPA 2-AC (Meta's tests in its own two labs, 2025-06) | Collapse-prevention recipes: EMA plus stop-gradient, SIGReg (2025-11), LeWM (2026-03); hierarchical planning with H-JEPA (2026-10) | As of 2026-10-15: whether AMI Labs releases a model; whether JEPA planning leaves lab arms and drops sub-goal images |
| Real-time interactive video worlds can be built: Genie 3 at 720p, 24 fps (DeepMind blog, 2025-08) | Memory and consistency beyond minutes: about 1 minute of visual memory for Genie 3 (DeepMind blog, 2025-08); several 2026 memory benchmarks all report systematic failures | Follow-up results on memory tests such as WRBench (2026-06) and WorldMemBench (2026-10) |
| World models can serve as policy evaluators with a reported correlation to real outcomes: a Veo-based evaluator at Pearson 0.86 for its strongest policy across several out-of-distribution axes (Google DeepMind, 2025-12) | World models as the policy itself: DreamZero (2026-02), Cosmos Policy (2026-01), 1X (2026-01), all tested by their own authors | As of 2026-10: whether driving simulators (GAIA-3/4, Waymo) publish correlations with real outcomes |
| Visual realism is not physical correctness, nor usefulness to an agent: Physics-IQ (2025-01), WorldArena (2026-02) | Physics-IQ Verified scores are rising, but the top entry used best-of-N and was vendor-submitted (leaderboard, read 2026-10-15) | Like-for-like comparisons; whether outcome distributions are calibrated (Odyssey's CaliBench, 2026-08, vendor-published) |
"Fairly settled" means a primary result holds in the original paper's setting; it does not mean it holds for other tasks or data.
How this relates to other notes on this site
- The dynamics models, MPC, compounding error and "planners exploit model flaws" in Model-based RL: prediction is not planning all still apply to world models; only the state has grown from a few numbers into images, video or embeddings.
- Inside planning: CEM, LQR, and learning in a model: V-JEPA 2-AC uses exactly the CEM explained there, and the "Latent imagination" row in that note's table is Dreamer.
- Which model to learn sorts models by whether they predict states, observations, latents or only values, which complements the six senses here; How planning and learning feed each other covers MCTS, AlphaZero and MuZero-style implicit planning.
How this series reads
- Learning inside imagination (Dreamer): RSSM, training actor-critic on imagined trajectories, a hand-computed λ-return, and how to read Dreamer 4's 0.7%
- Predict representations, not pixels (JEPA): why not predict pixels, representation collapse and its fixes, and how V-JEPA 2-AC plans with embedding distances
- Video you can act in (Genie and playable worlds): real-time interactive video world models such as Genie
- For robots and cars: simulator, evaluator, and policy: world models as data engines, as policy evaluators, and as the policy itself
- Do they actually understand the world?: what probes, physics benchmarks and memory tests show
- Review questions: interview questions and a self-check