"Language models often use relevant information more reliably when it appears near the beginning or end of a long context than when it appears in the middle."
Liu and colleagues measure how language-model performance changes with the position of relevant information in long inputs. Their experiments document the lost-in-the-middle effect, showing why making information available inside a context window does not guarantee that a model will use it consistently.
View Source