Format Online paper / PDF
Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER Code as a Coordination Layer

"Ambiguous natural-language requirements can reduce the correctness of LLM-generated code and produce divergent implementations."

Yang and colleagues introduced Orchid, a benchmark of 1,304 function-level tasks involving lexical, syntactic, semantic, and vague requirements. Their experiments found consistent performance degradation under ambiguity and frequent functional disagreement between generated implementations, demonstrating the importance of specification quality in agentic programming.

View Source
Source title Clarity Is Not Assumed: Understanding LLM-Based Code Generation under Ambiguous Requirements
Source authors Di Yang, Xinou Xie, Xiuwen Yang, Ming Hu, Yihao Huang, Yueling Zhang, Weikai Miao, Ting Su, Chengcheng Wan, Geguang Pu
Source year 2026
Confidence high
Importance high
Back to evidence PREVIOUSNEXT