"Even formally valid mathematical benchmarks can contain discrepancies between their formal statements and the informal problems they are intended to represent."
The researchers compared informal mathematical problems with their Lean formalizations and identified discrepancies in more than half of the examined miniF2F problems. Their corrected benchmark changed the results of the complete formalization-and-proving pipeline, directly illustrating the gap between proving a formal statement and proving the intended proposition.
View Source