"Baldur Bjarnason argues that language models are constructed from distributions in human writing, and that their outputs can correspond to reality partly because written observations correlate with the world."
Bjarnason, author of Out of the Software Crisis, characterizes language-model training as a process of constructing a mathematical artifact from textual data. He argues that factual correspondence can emerge through the indirect relationship between written language, human observations and reality, reinforcing the distinction between a textual map and the territory it describes.
View Source