"Addressing hallucination incentives requires changes to the widely used evaluation infrastructure, not only the creation of new specialised tests."
Kalai and colleagues describe the persistence of hallucinations through post-training as a socio-technical problem shaped by prevailing benchmark incentives. Their proposal is to change the scoring of influential evaluations so that model-development incentives reward appropriate uncertainty.
View Source