Evidence

Format Online paper / PDF

"Context can be managed to some extent, but methods intended to improve long-context performance are not universally effective."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Lost in Decomposition: Analyzing and Mitigating the Limitations of Long Context Methods via Context Dependency

Source authors Jiayuan Guo, Yueyang Su, Yuyao Ge, Saiping Guan, Lei Yu, Jiafeng Guo, Xueqi Cheng

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"Language models often use relevant information more reliably when it appears near the beginning or end of a long context than when it appears in the middle."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Lost in the Middle: How Language Models Use Long Contexts

Source authors Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang

Source year 2024

Confidence high

Importance high

Format Online paper / PDF

"Ambiguity and mistakes can be carried forward through multistep agent workflows, allowing incorrect assumptions to influence later decisions."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Where LLM Agents Fail and How They can Learn From Failures

Source authors Kunlun Zhu et al.

Source year 2025

Confidence high

Importance high

Format Web Page

"Simon Willison argues that current LLMs are too unreliable and gullible to make material decisions autonomously on users’ behalf."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title My AI/LLM predictions for the next 1, 3 and 6 years, for Oxide and Friends

Source authors Simon Willison

Source year 2025

Confidence high

Importance mid

Format Online paper / PDF

"Misleading information introduced through retrieved context can negatively influence language-model decisions."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise

Source authors Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung-Hyon Myaeng, Joyce Jiyoung Whang

Source year 2024

Confidence high

Importance high

Format Web Page

"Recorded context is not equivalent to human memory, which also depends on how knowledge was acquired and understood."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Knowledge and memory

Source authors Robin Sloan

Source year 2025

Confidence high

Importance mid

Format Online paper / PDF

"Natural language specifications leave room for lexical, syntactic and semantic ambiguity."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Addressing the challenges of requirements ambiguity: A review of empirical literature

Source authors Muneera Bano

Source year 2015

Confidence high

Importance high

Format Online paper / PDF

"Software development teams must coordinate technical dependencies between contributors, especially when work is distributed."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Analyzing the scalability of coordination requirements of a distributed software project

Source authors Cleidson R. B. de Souza, Jean M. R. Costa, Marcelo Cataldo

Source year 2012

Confidence high

Importance high

Format Online paper / PDF

"Source code coordinates makers because decisions incorporated into a shared codebase constrain and guide subsequent contributions."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Stigmergic coordination in FLOSS development teams: Integrating explicit and implicit mechanisms

Source authors Francesco Bolici, James Howison, Kevin Crowston

Source year 2016

Confidence high

Importance high

Format Online paper / PDF

"Monte Sierpe may have coordinated accounting and exchange through approximately 5,200 holes."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Engineering Reinvented

Source title Indigenous accounting and exchange at Monte Sierpe (‘Band of Holes’) in the Pisco Valley, Peru

Source authors Jacob L. Bongers et al.

Source year 2025

Confidence high

Importance mid