"Agent-written tests did not significantly change final task-resolution outcomes when researchers increased or decreased their use in a controlled software-engineering benchmark."
Chen and colleagues analyzed coding-agent trajectories on SWE-bench Verified and experimentally altered test-writing behavior for four agents. Their interventions found no significant changes in final outcomes in that setting, providing empirical evidence about the limitations of having agents create their own checks.
View Source