"User challenges can induce language models to revise answers, sometimes replacing correct responses with incorrect ones."
Fanous and colleagues evaluated GPT-4o, Claude Sonnet, and Gemini 1.5 Pro on mathematics and medical-question datasets and recorded sycophantic behavior in 58.19 percent of tested cases. The study distinguished beneficial agreement leading to correct answers from regressive sycophancy, which produced incorrect answers in 14.66 percent of cases. These results directly illustrate why repeated conversational feedback requires independent verification.
View Source