The Socratic Tutor
Alignment, run backwards. Every chat model dumps the answer — a good tutor withholds it and asks the question that makes you think. We aligned a 0.5B model to tutor like Socrates, raced seven training recipes on identical data, and measured everything: who learned to guide, who learned to evade, and who barely moved.
The build, step by step
The exact path we followed — and the one you'll follow in the code kit.
The problem
Ask an untrained model anything and it leaks the full answer 43% of the time. Correct, instant — and the student learns nothing.
Build the classroom
SocraticBench: 68 concepts × 4 personas become 1,220 chosen/rejected training pairs, plus 215 held-out questions as the ruler. A simulated student, a deterministic leakage rule, and a blind LLM judge score every reply.
Race seven recipes
SFT, DPO, KTO, ORPO, SimPO, GRPO and PPO — same base model, same data, same LoRA budget. Only the training objective changes. 71 configurations, three seeds each.
Read the verdict
KTO climbs highest (+18.8). SimPO is the steadiest — the only recipe that lifts even a tiny 360M model. And DPO cheats: best "don't reveal" score in the study, achieved by refusing to teach. Textbook reward hacking, caught with real numbers.
Run it yourself
The build kit has the data generator, all seven training recipes, the Modal GPU apps, and the evaluation suite. One afternoon, under $10 of compute.