Quantization from Scratch
From uniform quantization to TurboQuant — compress LLMs from first principles without losing accuracy.
intermediate~4 hours2 pods live
Pods in this Course
1
TurboQuant: How Google Compressed AI Memory by 6x Without Losing a Single Answer
How Google compressed KV cache memory by 6x using random rotations and Johnson-Lindenstrauss projections — without losing accuracy.
~4h3 notebooksCase study
2
MXFP4 Quantization-Aware Training from Scratch
Block-scaled 4-bit floating point (MXFP4) quantization-aware training: why frontier open models like Kimi K3 ship 4-bit weights, how block scaling works (MXFP4 weights + MXFP8 activations, per-block scale factors), and why learning to compensate for quantization error during SFT beats post-hoc PTQ. From-scratch build: implement block-scaled quantization, a fake-quant QAT training loop, and compare PTQ vs QAT on a small LM.
~5h9 notebooksCase study