"Our theory of language models is still underdeveloped in the messy middle between pure noise and perfectly structured data."
In his December 2025 conversation with Brian Keating, Terence Tao explains that the mathematics used to build language models is comparatively accessible, while predicting their capabilities remains difficult. Around the 38-minute mark, he compares partially structured natural language with intermediate-scale phenomena in physics, where theoretical understanding is less developed.