"Simon Willison highlighted varying load and batch size as a key source of nondeterminism in LLM inference endpoints."
In his September 2025 commentary, Simon Willison points readers to Thinking Machines Lab's investigation and reproduces its finding that varying request load changes batch size and output reproducibility. His note is a readable secondary entry point to the original technical account.
View Source