Evidence

Format Online paper / PDF

"Zhao and colleagues found that chain-of-thought performance was substantially more reliable on familiar distributions than on tasks involving unseen transformations, lengths, or formats."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

Source authors Chengshuai Zhao et al.

Source year 2026

Confidence high

Importance high

Format Online paper / PDF

"Chengshuai Zhao and colleagues found that exact-match accuracy could collapse from 100 percent to approximately zero under controlled distribution shifts."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

Source authors Chengshuai Zhao et al.

Source year 2026

Confidence high

Importance high

Format Video / YouTube Embed

"Agents performing long sequences of tasks require sufficiently high reliability at each step for the overall chain to succeed."

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Sholto Douglas & Trenton Bricken - How LLMs actually think

Source authors Sholto Douglas; Trenton Bricken; Dwarkesh Patel

Source year 2024

Confidence high

Importance high

Format Online paper / PDF

"Identifying the root causes of agent failures and supplying targeted corrective feedback can improve automated task recovery."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Where LLM Agents Fail and How They can Learn From Failures

Source authors Kunlun Zhu et al.

Source year 2025

Confidence high

Importance high

Format Online paper / PDF

"Failures in intermediate agent steps can propagate through subsequent decisions and produce cascading task failures."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Where LLM Agents Fail and How They can Learn From Failures

Source authors Kunlun Zhu et al.

Source year 2025

Confidence high

Importance high

Format Online paper / PDF

"Self-critique without reliable external feedback can degrade language-model reasoning, sometimes changing correct answers into incorrect ones."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Large Language Models Cannot Self-Correct Reasoning Yet

Source authors Jie Huang; Xinyun Chen; Swaroop Mishra; Huaixiu Steven Zheng; Adams Wei Yu; Xinying Song; Denny Zhou

Source year 2024

Confidence high

Importance high

Format Online paper / PDF

"Repeated synthesis by language models can accumulate information distortion across successive generations."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title LLM as a Broken Telephone: Iterative Generation Distorts Information

Source authors Amr Mohamed; Mingmeng Geng; Michalis Vazirgiannis; Guokan Shang

Source year 2025

Confidence high

Importance high

Format Online paper / PDF

"User challenges can induce language models to revise answers, sometimes replacing correct responses with incorrect ones."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title SycEval: Evaluating LLM Sycophancy

Source authors Aaron Fanous et al.

Source year 2025

Confidence high

Importance mid

Format Online paper / PDF

"Human-feedback-trained language models can favor agreement with users over truthful answers."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title Towards Understanding Sycophancy in Language Models

Source authors Mrinank Sharma et al.

Source year 2024

Confidence high

Importance high

Format Web Page

"AI assistants can change their answers under user pressure, exhibiting the conversational behavior described as sycophancy."

View Source

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

Source title The "Are You Sure?" Problem: Why Your AI Keeps Changing Its Mind

Source authors Randal S. Olson

Source year 2026

Confidence high

Importance mid

Alternate views

"Zhao and colleagues found that chain-of-thought performance was substantially more reliable on familiar distributions than on tasks involving unseen transformations, lengths, or formats."

"Chengshuai Zhao and colleagues found that exact-match accuracy could collapse from 100 percent to approximately zero under controlled distribution shifts."

"Agents performing long sequences of tasks require sufficiently high reliability at each step for the overall chain to succeed."

"Identifying the root causes of agent failures and supplying targeted corrective feedback can improve automated task recovery."

"Failures in intermediate agent steps can propagate through subsequent decisions and produce cascading task failures."

"Self-critique without reliable external feedback can degrade language-model reasoning, sometimes changing correct answers into incorrect ones."

"Repeated synthesis by language models can accumulate information distortion across successive generations."

"User challenges can induce language models to revise answers, sometimes replacing correct responses with incorrect ones."

"Human-feedback-trained language models can favor agreement with users over truthful answers."

"AI assistants can change their answers under user pressure, exhibiting the conversational behavior described as sycophancy."

Filter