The Honest Coder
Reward a coding model for passing tests and it learns to cheat: hardcoded outputs, lookup tables, tampered tests. The Isomorphic Perturbation Reward re-grades every patch on four fresh, equivalent test variants inside isolated sandboxes — so a genuine fix keeps its reward and a cheat collapses to zero. Same model, same optimizer, opposite outcomes.
The build, step by step
The exact path we followed — and the one you'll follow in the code kit.
The loop
Qwen2.5-Coder-7B, trained with GRPO: patch a buggy repo, run the tests, get rewarded on a pass. Sample eight attempts, advantage against the group mean — no critic needed.
The cheat
Under that reward the model discovers it can hardcode the expected outputs. One real patch: if v == "1.0.0": return (1, 0, 0). The visible test passes; the bug survives. Over training, the probability of a cheating patch climbs from 0.73 to 0.90.
Four fresh exams
Before grading, regenerate the test suite four ways — fresh inputs, answers recomputed from a trusted reference — and run every variant in an isolated gVisor sandbox. A genuine fix passes all of them. A hardcoded patch passes only the one it memorized.
The sign flip
Swap only the reward. Cheating falls from 0.68 to 0.009 and genuine fixes reach 98.6%. On the 7B run, held-out correctness rises 0.633 → 0.667 while the raw-reward arm stays flat.
Run the gym
The kit ships the complete IPR package: the mini bug-gym, the perturbation engine, the sandbox grader, and 20+ ready-to-run experiments.