Context Window Overflow & Compaction

1. Concept Overview & Systemic Problem

In long-running engineering projects, an agent may work for hours: reading files, running tests, retrieving logs, and fixing bugs. Sooner or later, even a 200,000-token window fills up:

• Naive approach: simply truncate the earliest messages. However, this also removes the user's initial instruction, causing the agent to suddenly "forget" the task it is solving.
• Another naive approach: halt operation and report an error, forcing the user to manually recount everything from the beginning.

Context Window Compaction is a technique for continuously compressing dialogue history on the fly, allowing the agent to maintain endless work sessions without crashing due to overflow errors and without losing initial goals.

2. Architectural Taxonomy & Mental Model

┌─────────────────────────────────────────────────────────────┐
│                 CONTEXT COMPACTION LIFECYCLE                │
├─────────────────────────────────────────────────────────────┤
│ 1. BUFFER MONITORING (Total Tokens: 165k / 200k Limit)      │
│    • Trigger Threshold: 80% window fill                     │
├─────────────────────────────────────────────────────────────┤
│                          │                                  │
│                          ▼ Trigger Compaction Worker        │
├─────────────────────────────────────────────────────────────┤
│ 2. PARTITIONING & ROLLING SUMMARIZATION                     │
│    ┌─────────────────────────────────────────────────────┐  │
│    │ [PINNED] System Prompt & Core Rules (Immutable)     │  │
│    ├─────────────────────────────────────────────────────┤  │
│    │ [SLICED] Old Turns 1–30 ➔ Distilled into Executive  │  │
│    │          State: "Auth configured, DB migrated"      │  │
│    ├─────────────────────────────────────────────────────┤  │
│    │ [KEPT RAW] Fresh Turns 31–40 (Exact detail for dev) │  │
│    └─────────────────────────────────────────────────────┘  │
├─────────────────────────────────────────────────────────────┤
│ 3. CONTEXT RE-ASSEMBLY: New Size = 45k tokens (Fresh room!) │
└─────────────────────────────────────────────────────────────┘

3. Technical Pipeline & Internal Mechanics

01. Infinite Vibe Coding Session in Agent IDE

A developer has been working in Cursor or Claude Code for the sixth consecutive hour. The system seamlessly compresses old terminal logs into a neat markdown block <previous_session_summary> every 40 messages, keeping the session fast, cost-effective, and responsive.

02. Preserving Decision Artifacts (Architectural Decision Records)

During compaction, the agent separately retains rejected hypotheses: "We tried using library X, but it is incompatible with Windows." This ensures that after 100 messages, the agent does not attempt to install library X again.

4. Pitfalls, Common Mistakes & Security

• Information Loss: The summarization model may decide that specific port numbers or names of rare fields are unimportant and remove them from the summary. Critical parameters should be stored in dedicated JSON State fields, not in free text.
• Goal Drift with Multiple Compactions: If a session lasts for weeks and the summary is compacted for the tenth time, the "telephone game" effect occurs. The user's initial request (User Intent) must always remain unchanged at the root of the prompt.

5. Strategic Conclusion for the Engineer of 2026

Context compaction is key to creating long-running autonomous systems (Long-Running Agents). The ability to design memory rotation and summarization algorithms enables the development of agents that can work on a project for weeks without losing focus.