Build a World Model from Scratch
The written companion to the Vizuara lecture series: what a world model really is, the four tools every one stands on, and your first complete build — an encoder, a memory and a prediction head that learn MiniPong well enough to become the game.
Pods in this Course
What Is a World Model, Really?
The agent–environment loop, why state is not observation, eighty years of the idea, and the map of today's field.
The World Modeler's Toolkit
The four tools every world model stands on: latent compression, reward over time, value, and actor-critic learning.
Your First World Model — MiniPong
Build the complete pipeline — encoder, GRU memory, prediction head — train it on MiniPong, run it as the game, and watch the dream collapse.
Dreams That Last — the RSSM on a Real Robot
Fix the collapsing dream: a state with two halves — deterministic memory and sampled doubt — trained on real SO-101 robot data.
A Vector or a Vocabulary? — IRIS and Discrete World Models
Turn frames into tokens and the world model into a transformer — IRIS built from scratch on Mario.
The Journey of a Token Through a World Model
Follow one token through the trained transformer — embedding, attention, the residual stream, and out to logits.