Explainer1 min read
What Is a Context Window and Why It Fills Up
The context window is the model’s working memory: everything it can consider when producing an answer. Windows have grown enormous, but filling them is rarely the best strategy.
TrueCodeAI Engineering
Agents, Voice & ML practice
- Published
LLMAI agentsExplainer
What goes in it
- System instructions and tool definitions.
- The conversation so far.
- Retrieved documents and tool results.
- The model’s own output as it writes.
Why bigger is not always better
- Cost: every token in the window is billed on every call.
- Latency: more input means slower responses.
- Attention: models can miss details buried in very long contexts.
A focused context of the right few thousand tokens usually beats a huge context of everything.
How agents manage context
- Retrieve only what the current step needs.
- Summarise or clear old tool results once used.
- Write progress and notes to files, and read back only what is needed.
- Split big tasks across sub-agents, each with a clean context.
Frequently asked questions
What happens when the window is full?
Older content must be dropped or summarised; otherwise the request fails. Good harnesses handle this automatically.
Should we paste whole documents in?
For one-off analysis, sometimes. For repeated questions, retrieval is cheaper and more accurate.
Does a larger window mean better reasoning?
No — it means more capacity to read. Reasoning quality depends on the model and on what you put in.
