Lost in the Middle Effect
A well-documented cognitive asymmetry of large language models identified in Stanford research. It demonstrates that the accuracy of information retrieval is highest at the beginning and end of the input context, but sharply declines in the middle of long documents.
1. Concept Overview & Systemic Problem
Have you ever tried to read a thick textbook in one night before an exam? Typically, you remember the introductory chapter and the last pages you flipped through before leaving home. However, what was on page 240 tends to blur into a gray mass.
Strangely enough, large language models behave exactly the same way!
In 2023, researchers from Stanford conducted an experiment: they loaded long lists of documents into models and hid the required answer in various locations. The result shocked the industry: if a fact was located in the middle of a long text, the model's chances of finding it dropped from 90% to 30%!
This phenomenon has been termed the Lost in the Middle effect.
The mental model: the most crucial practical secret is that the location of information in the prompt is critical.
2. U-Shaped Attention Curve of the Model
FACT RETRIEVAL ACCURACY
100% ────┐ ┌──── 100%
80% │ ╲ ╱ │ 80%
60% │ ╲ ╱ │ 60%
40% │ ╲ ╱ │ 40%
20% │ ╲═════════════════════════════════╱ │ 20%
│ "DEAD ZONE OF THE MIDDLE" │
0% ────┴───────────────────────────────────────────┴──── 0%
BEGINNING OF TEXT MIDDLE END OF TEXT
(System Prompt) (Do not place anything (Question)
important here!)
3. How to Structure Long Queries Correctly
❌ Faulty Structure (Risk of Failure):
1. [Long 50-page document]
2. [Somewhere in the middle on page 25: "Main rule: respond only in Ukrainian"]
3. [Question about the text]
Result: The model will miss the rule on page 25 and respond in English.
✅ Flawless Engineering Structure:
1. [Clear instruction at the beginning: "Analyze the provided document. Respond strictly in Ukrainian."]
2. [Array of the document in delimiters <document> ... </document>]
3. [Reminder at the very end: "Remember: extract only 3 key points and write in Ukrainian."]
4. Production Engineering Scenarios
01. Long Document Analysis
When analyzing lengthy documents, ensure critical instructions are placed at both the beginning and end to avoid the middle forgetting effect.
02. Multi-Section Queries
For queries spanning multiple sections, repeat essential prompts to maintain context and ensure accurate responses.
03. User Interaction Design
Design user interactions to emphasize key information at the start and end, minimizing reliance on the middle content.
5. Pitfalls, Common Mistakes & Security
Avoid hiding key requirements or critical facts deep within a vast text. Always highlight the most important information in the first and last 5% of your message.
FAQ: Lost in the Middle Effect
Related terms
Needle In A Haystack Test (NIAH)
A standard benchmark for testing the context window length of language models. A short random sentence (the needle) is hidden within a vast array of random background text (the haystack) at various positions, assessing the model's ability to locate it accurately.
Context Window Size (Current Conversation Memory)
The maximum amount of text (in tokens) that a language model can simultaneously retain in memory during an ongoing conversation. It determines the length of documents that can be loaded at once without loss of content.
Delimiters and XML Tags (Structuring Prompts)
A technique for formatting complex prompts using XML tags (<context>, <rules>) and triple quotes (\"\"\"). It helps the model clearly distinguish where your rules end and where the text for processing begins.