"In Dex Horthy’s three-problem SlopCodeBench experiment, Opus 5 passed four of seventeen strict checkpoints, while the other two evaluated models passed one each."
Horthy's July 2026 report describes an incremental evaluation of three Claude models using held-out black-box tests and regression checks from earlier stages. Even the strongest model cleared only four of seventeen checkpoints, illustrating the challenge of sustaining correctness across successive implementation tasks.
View Source