"Human programmers can struggle to recognise incorrect LLM-generated assertions even when they feel confident in their judgments."
In an experiment involving 86 Python programmers, Kaufman and colleagues found 74% accuracy when evaluating correct generated assertions but only 49% when evaluating incorrect ones. Similar confidence across both conditions demonstrates a practical obstacle to relying on human inspection without stronger verification methods.
View Source