"Standard training and many accuracy-focused evaluations reward guessing over acknowledging uncertainty."
OpenAI's paper argues that hallucinations persist partly because scoring systems encourage models to produce answers even when they are uncertain. Under binary accuracy grading, a lucky guess can improve the score while an honest abstention earns nothing.
View Source