"Influential AI benchmarks are often framed as general measures of progress despite serious construct-validity problems."
Raji and colleagues examine how finite, contextual benchmarks become stand-ins for broad claims about intelligence and generalization. Their analysis supports the distinction between measured performance and the wider capability attributed to a model.
View Source