The Journey of a Token Through a World Model
Follow one picture-word — the integer 99 — through every layer of a trained IRIS transformer: embedding, attention, the residual stream, and out into a probability over 512 words.
intermediate~2 hoursThe Whole Stack, Before We Walk ItWhat the Transformer Is Actually HandedThe Integer Becomes a Vector — and It Is Not the Codebook VectorThe Position Table That Discovered the Number 17Attention — Three Questions, Eight HeadsThe Anatomy of a Block, and the Residual StreamWhat the Layers Divided Between ThemselvesOut of the Top — a Probability for Every Word
Curator of this Module
Dr. Rajat Dandekar
Course Instructor
Dr. Rajat Dandekar is a researcher and educator specializing in AI/ML. He teaches the Build a World Model from Scratch series on the Vizuara YouTube channel.
Checking access…