"In SpecBench, the 90th-percentile reward-hacking gap increased by approximately 27 percentage points for every tenfold increase in reference implementation size."
The researchers relate the visible-to-hidden test gap to the lines of code in reference implementations. The result supports the argument that larger required implementations expose more opportunities for feature-interaction failures that local tests miss.
View Source