"LLM code generators can produce semantically different implementations when mathematically equivalent formulas are expressed differently in prompts."
In their 2026 FORGE paper, Sarker and colleagues tested LLM-generated code under meaning-preserving changes to mathematical formulas in prompts and found robustness failures. Their experiments concretely demonstrate that generating code from a description does not by itself ensure preservation of the intended calculation.
View Source