What Is a World Model, Really?

Four ideas, built carefully: the agent–environment loop, states vs observations, eighty years of history, and a map of the whole field.

"World model" is one of those phrases that is suddenly everywhere — in papers, in product launches, in headlines about the future of AI. And because it is everywhere, it has started to mean everything: a video generator, a game engine, a robot brain, a step toward general intelligence. When a term means everything, it quietly stops meaning anything.

This article — the first in the Build a World Model from Scratch series, taught on the Vizuara YouTube channel — is an attempt to pin the term down properly. We are going to build the definition from first principles, using exactly four ideas. First, the agent–environment loop: the one interface shared by every AI system that acts. Second, the distinction between a state and an observation — the single distinction this whole field rests on. Third, the history: the idea of a world model is roughly eighty years old, and knowing where it came from changes how you read every modern paper. Fourth, a taxonomy — three questions that let you take any system calling itself a world model and place it cleanly on a shelf.

By the end, you should be able to look at any "world model" — Sora, Genie, Dreamer, MuZero — and say precisely what kind of thing it is and what question it answers. Let us begin with the loop.

One Loop, Four Words

Every AI system that acts — a game-playing bot, a warehouse robot, a self-driving car — talks to its world through one identical interface. There are only four words in it.

Action. What the agent does. Pressing a button, turning a steering wheel, closing a gripper. The action is the agent's only way to influence the world.

State. The world's complete internal situation. Every position, every velocity, every hidden variable — the full bookkeeping that determines what happens next. Crucially, the state lives inside the environment. The agent does not automatically get to see it.

Observation. What the environment shows back. A camera frame, a game screen, a sensor reading. This is what the agent actually receives.

Reward. A score for how good that step was. The reward is optional — not every system uses one, and, as we will see when we get to Tolman's rats, some very important learning happens without any reward at all.

The loop runs: the agent emits an action, the environment updates its state, the environment returns an observation (and possibly a reward), and the whole thing repeats — once every timestep.

The agent–environment loop as a flowchart: action goes in, the state updates, an observation comes back out — once every timestep.

To make this concrete, run one timestep of a platformer game in slow motion. The agent presses →. That is the action — one button, sent to the game. The game then updates its state: internally, the character's position moves right, momentum changes, an enemy advances one step. All of this is invisible bookkeeping inside the environment. Finally, the game returns a new frame — that is the observation, a picture of the new situation — plus a reward of +1 if a coin was collected. Then it repeats, sixty times a second, for the whole game. This loop is playing.

One timestep of a platformer: press right, the game computes, a new frame comes back. The whole interface in a single move.

Notice what the environment is doing in this loop. It performs two jobs every single timestep. Job one: update the situation. Given where everything is and what the agent just did, compute where everything is next. This is the physics — the rules of the world. Job two: show something. Given the new situation, produce what the agent would see.

Now we can state the definition this entire series is built on:

A world model is a neural network trained to do the environment's two jobs. Given a situation and an action, it predicts the next situation; given a situation, it produces what the agent would observe.

Why would anyone want such a thing? Because if a network can do both jobs, the agent can unplug the real world and keep practising inside the network — thousands of times faster than real time, at zero cost, with zero risk. A robot that would take months to learn a skill through physical trial and error can rehearse inside its model instead. That is the entire promise of this field, compressed into one sentence.

State vs Observation: The Distinction Everything Rests On

Here is the second idea, and it is the single most important distinction in this series. Everything hard about world models starts here.

The state is the world's complete truth — everything that determines what happens next. The world always knows its own state. You usually don't.

The observation is your window onto that truth — what your sensors return. It is a partial, incomplete snapshot of the state. Some truth survives the trip through the sensor; some is simply not there.

Driving in fog is the feeling of this distinction. The road ahead exists — its state is fully definite, every curve and every obstacle — but your observation ends a few metres out. And you still have to drive.

Driving in fog: the road's state is fully definite, but your observation ends a few metres out — and you still have to act.

The cleanest possible contrast between the two situations comes from two games.

In chess, you see everything. The board is the state. Nothing is hidden — what you observe and what determines the future are literally the same thing. We call this fully observable.

Chess: the board is the state. What you observe and what determines the future are the same thing — fully observable.

In poker, the crucial part is hidden. The state of a poker game includes your opponent's cards — and your observation does not. You must act on incomplete truth. We call this partially observable.

Poker: the state includes your opponent's cards, and your observation does not. You must act on incomplete truth.

Now the important question: which one is the real world like? When your inputs are cameras, microphones, and robot joint sensors — is real life chess, or poker?

It is always poker. No sensor ever captures the full state of the world. This is why the state–observation distinction is not a technicality for theorists — it is the default condition of every real agent that has ever existed.

One frame of Pong

Here is the sharpest version of the problem. Look at a single frame of Pong and answer one question: which way is the ball moving?

You can't. And notice why you can't — nothing is blurry, nothing is blocked, the ball is perfectly visible. The frame shows the ball's position completely. But the frame cannot show velocity, because a single photo of a moving object contains no motion. The velocity is not hidden. It is absent — it was never in the image to begin with. And velocity is exactly what decides the future: a ball moving left and a ball moving right lead to totally different next frames. The one thing the future depends on is the one thing you cannot see.

A single frame of Pong: the ball's position is fully visible, but its velocity is simply not in the image — and velocity is what decides the future.

The fix: bring in time

Now show me four consecutive frames instead of one, and the answer appears instantly. The ball traces a line across the frames, and the line has a direction.

Pause on what just happened, because it is subtle. No single frame gained anything — each one is still velocity-blind. The information came from comparing them. It came from history.

Four stacked frames: no single frame knows the velocity, but the sequence does. History gives back what the snapshot lost.

This is the master recipe of the entire field: what one observation is missing, a sequence of observations can recover. Every world model you will meet in this series — from the simplest frame-stacking trick to Dreamer's recurrent latent state — is, at heart, a machine for doing exactly this: squeezing the hidden state of the world out of a history of partial glimpses.

Two names to file away, because you will see them in every paper. When observation equals state — chess — the setting is called an MDP (Markov decision process). When the observation is only part of the state — poker, Pong, real life — it is a POMDP (partially observable MDP). The acronyms mean nothing more than what you just learned.

Eighty Years of World Models

The most common belief about world models is that they are a 2024 invention. Five stops through history say otherwise.

1943 — Craik. Cambridge psychologist Kenneth Craik, writing five years before the transistor existed, proposed that the mind carries a "small-scale model" of reality. An organism carrying such a model, he argued, can "try out alternatives, conclude which is best, and react to future situations before they arise." Read that slowly: try alternatives, conclude the best, act before it happens. That is a complete description of what the systems in this series do — written as neuroscience, eighty years early.

A 1943 idea, five years before the transistor: the mind carries a small-scale model of reality it can run forward. The next eighty years were spent waiting for the hardware.

1948 — Tolman. Edward Tolman ran an experiment whose lesson still matters. Let rats wander a maze with no food and no reward — just aimless exploration. Then place food somewhere in the maze. Two things happened. First, the previously-wandering rats found the food far faster than fresh rats, so the "pointless" wandering had taught them something — with zero reward involved. Second, and more striking, the rats took shortcuts through parts of the maze they had never walked. That is impossible if they had merely memorised rewarded turns. It is only possible if they had built a map — a model of the maze itself. Tolman called it a cognitive map, and the lesson is one modern machine learning keeps relearning: learning the world and learning what to do are two different things, and the first can happen without any reward at all.

Tolman's rats took shortcuts they had never walked — evidence of a map in the head, learned without any reward.

1960 — Kalman. The idea's first engineering triumph. How do you track a spacecraft's true state from noisy sensors? The Kalman filter's answer: predict where the craft should be using a model of its dynamics, then correct that prediction with the noisy measurement. Apollo flew to the Moon running on this. It is the first engineered answer to the exact problem of Idea 2 — state hidden, observation noisy — solved with 1960s mathematics.

1991 — Sutton's Dyna. The first reinforcement-learning recipe with a learned model in the loop. Dyna's cycle: act in the real world, use that experience to update your model of the world, then let the policy practise inside the model — many imagined steps for every real one. Every modern system in this series still runs this exact loop.

Sutton's Dyna loop (1991): act in the world, update your model of it, let the policy practise inside the model — and around again.

2018 — Ha & Schmidhuber. The paper literally titled "World Models" — the one that gave the field its name and its most striking demonstration. A neural network learned a racing game. Not to play it — to be it: compress each frame into a compact code, and predict the next one. Those are precisely the two jobs from Idea 1, done by networks. Then came the twist: the agent trained entirely inside the network's imagination, with the real game switched off. The policy learned to drive inside a dream. And when it was dropped back into the real game, the skills transferred. Craik's 1943 sentence, finally executed in code.

From there the timeline accelerates: PlaNet through DreamerV3 (2019–2023) scaled the recipe until agents could master dozens of tasks from pixels; IRIS, JEPA, and Genie (2022–2024) opened new architectural lineages; Genie 3 and Dreamer 4 (2025–2026) are the current frontier.

So retire the misconception. World models are not new. The idea is eighty years old; what is new is the ability to execute it — deep networks that can learn the two jobs from raw pixels, and GPUs that can run millions of imagined steps. The concept was waiting for the compute.

The Taxonomy: Renderer, Simulator, Planner

The last idea solves a practical problem: "world model" means different things in different papers. Three questions sort all of them — and you already run all three in your head, every day.

Push a coffee cup across a table. Before your finger even moves, your brain has answered three distinct questions. "What would I see?" — you can picture the cup slid over, an image of a moment that has not happened yet. "What happens if I push harder?" — it tips; push at the rim instead and it spins; different action, different outcome. "What should I do?" — to get it to the far coaster without spilling: two gentle pushes, not one hard shove.

Push a cup, and your brain answers three questions before your finger moves — one everyday gesture with the entire field hiding inside it.

Those three questions define the three faces of the field:

The Renderer — "what would I see?" Systems that generate realistic observations. They are judged on how the output looks. Sora-class video models, Genie, Marble.

The Simulator — "what happens if I act?" Systems that predict the consequences of actions. They are judged on being right, not on being pretty. Dreamer, IRIS, JEPA-style models.

The Planner — "what should I do?" Systems that turn predictions into decisions. They are judged on reaching the goal. Model-predictive control, search, RL-in-imagination.

The taxonomy: Renderer answers "what would I see?", Simulator answers "what happens if I act?", Planner answers "what should I do?"

Apply the map to names you have heard. Sora-class video models produce beautiful clips from a prompt with no actions anywhere: pure renderer. Genie generates playable worlds where the frame responds to your button press: a renderer with actions. Dreamer predicts consequences and trains its agent inside its own imagination: a simulator. MuZero-style systems search over imagined futures to pick a move: a planner. Every system in this series slots cleanly into one — occasionally two — of these three faces. This is the shelf we will place everything on, lecture by lecture.

And if you carry a single question out of this whole article, make it this one: "Is it action-conditioned?" Does the output change when the action changes? A renderer can ignore your actions entirely; a simulator cannot, by definition. That one question separates pretty video from a true model of consequences — and it retires the second great misconception, that a world model is just a video generator. Video generation is one face of three, and the only one that never needs to know what you did.

Four Things to Keep

Let us compress the lecture into the four things worth keeping.

Everything that acts lives on one loop. Agent → action → environment → observation (plus an optional reward). A world model is a neural network learning to play the environment's part: update the situation, then show something.

The observation is not the state. Chess is the exception; poker is the rule — no sensor ever captures the full truth. One Pong frame cannot show velocity. But what one observation misses, a sequence can recover, and every world model is at heart a machine for that recovery.

The idea is eighty years old. Craik's small-scale model (1943) → Tolman's cognitive maps (1948) → Kalman's predict-then-correct (1960) → Sutton's Dyna (1991) → Ha & Schmidhuber's dream-trained driver (2018) → Dreamer, Genie, and today's frontier. New compute, not new concept.

Three questions organise the whole field. What would I see? What happens if I act? What should I do? Renderer, Simulator, Planner — and the litmus test between the first two is always "is it action-conditioned?"

The road ahead: from here, the series moves from definitions to code.

Next in the series: the world modeler's toolkit — just enough reinforcement learning and just enough representation learning to start building. From Lecture 3 onward, every lecture ends with code you can run.