"OthelloGPT, trained to predict legal Othello moves from move sequences, developed a causally relevant internal representation of the board state."
Li and collaborators trained a transformer without directly presenting the board or rules and used probing and interventions to find internally encoded board-state information. First released as a 2022 preprint and published at ICLR 2023, it supports the initial world-model evidence.
View Source