"At SWE-Lancer’s 2025 introduction, the frontier models tested could not solve the majority of its freelance engineering tasks."
The researchers found that the tested frontier systems failed on most tasks despite their progress on conventional programming benchmarks. The historical result supports the distinction between benchmark proficiency and independently completing economically meaningful engineering assignments.
View Source