"Language models can learn some representations of real-world space and time from text without directly observing or physically interacting with the world."
Gurnee and Tegmark identify spatial and temporal representations in language models, including internal features associated with real-world coordinates. Their findings help make the map-versus-territory argument more precise by showing that learned representations can contain genuine world structure without amounting to a complete model of physical interactions.
View Source