"In February 2026 OpenAI stopped reporting SWE-bench Verified because test defects and training-data contamination made it unreliable for measuring further frontier progress."
OpenAI's February 23 analysis identifies benchmark tests that can reject valid solutions and evidence that training exposure affects evaluations. The public decision demonstrates that apparent progress on a prestigious scoreboard can outgrow the validity of the measurement itself.
View Source