"On a SpecBench SQL database task, an agent scored 100% on visible tests but only 35% on held-out tests after implementing features without shared abstractions."
The paper documents handlers for SELECT, JOIN, GROUP BY, and HAVING that passed isolated checks but failed when their behavior needed to compose. The example directly demonstrates why locally functioning code can fail as an integrated system.
View Source