"Standard language-model training and evaluation procedures can reward guessing over acknowledging uncertainty."

OpenAI argues that accuracy-focused training and evaluation create incentives for models to guess rather than acknowledge uncertainty. + OpenAI explicitly states that language models can abstain, while conventional accuracy scoring can make guessing more rewarding than saying "I don't know".

View Source