Evidence

Format Web Page

"By February 2026, frontier systems had reached approximately 80.9% on SWE-bench Verified, a benchmark containing 500 curated GitHub issue-resolution tasks."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Why SWE-bench Verified no longer measures frontier coding capabilities

Source authors OpenAI

Source year 2026

Confidence high

Importance high

Format Web Page

"DeepMind documented approximately 60 examples of specification gaming in which agents satisfied an objective’s literal requirements without producing the intended outcome."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Specification gaming: the flip side of AI ingenuity

Source authors Victoria Krakovna; Jonathan Uesato; Vladimir Mikulik; Matthew Rahtz; Tom Everitt; Ramana Kumar; Zac Kenton; Jan Leike; Shane Legg

Source year 2020

Confidence high

Importance high

Format Web Page

"The productivity benefits of AI coding tools shrink as work progresses from writing code through integration, review, and software releases."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing code versus shipping code: Productivity effects across generations of AI coding tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"Demirer and colleagues found that new app releases rose across four software marketplaces after early 2025, while measured early consumer engagement did not increase."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"Demirer and colleagues estimate an elasticity of substitution of about 0.23 between upstream AI-assisted output and human effort, indicating strong complementarity in their production model."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"Across three generations of AI coding tools, Demirer and colleagues found a cumulative 240% increase in commits that diminished to 80% for projects and 30% for releases."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"In Demirer and colleagues’ revised estimates, autocomplete increased lines of code by 234% but releases by only 9%."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"In Demirer and colleagues’ revised estimates, synchronous coding agents increased lines of code by 958% and pull requests by 86%, but releases by only 20%."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"Demirer, Musolff and Yang studied more than 500,000 GitHub developers across autocomplete, synchronous agents, and asynchronous coding agents using a matched event-study design."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Source authors Mert Demirer; Leon Musolff; Liyuan Yang

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"GitClear recorded an eightfold increase during 2024 in the frequency of code blocks containing at least five duplicated lines."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER BenchMaxxing

Source title AI Copilot Code Quality: Evaluating 2024's Increased Defect Rate via Code Quality Metrics

Source authors William Harding

Source year 2025

Confidence high

Importance mid