VLMs from Scratch
Build vision-language models from first principles — from ViT to multimodal instruction tuning.
Pods in this Course
Vision Encoders: How Machines Learned to See -- From Convolutions to Vision Transformers
Understanding the two paradigms of visual representation learning -- from local feature extraction with CNNs to global attention with Vision Transformers.
Vision Transformers from Scratch: How Treating Images as Sentences Changed Computer Vision
We break down the Vision Transformer (ViT) paper step by step — from image patches to self-attention — with intuition, math, and a full PyTorch implementation.
Multimodal Fusion Architectures: How AI Learns to See, Read, and Listen — All at Once
From early fusion to cross-attention — building the bridges that connect vision and language inside modern AI systems.
Contrastive Pretraining (CLIP-style): Teaching Machines to See and Read at the Same Time
How CLIP learned to connect images and text in a shared space -- from first principles to implementation.
Cross-Attention & Token Alignment: How Vision-Language Models Learn to See and Speak
Understanding the mechanism that allows language models to look at images - from first principles to a working implementation.
Multimodal Instruction Tuning: Teaching Language Models to See and Think
How LLaVA-style visual instruction tuning transforms a language model into a multimodal reasoner -- from projection layers to two-stage training to cross-modal attention.