"Misra and collaborators trained transformers on hidden modular recurrence rules that predicted well at supervised positions but failed at the first unsupervised position."
The team compared rule-generated with random number sequences and restricted training loss to positions one through five. The model identified the rule very precisely at positions three to five but failed at position six, illustrating identification without reliable positional reuse in this controlled experiment.
View Source