Treat AI context as a budget to manage
Canonical version: Treat AI context as a budget to manage.
The Context Window is a BUDGET, not a capacity to fill. Every token in context costs money (you pay for it again on every turn), latency, and attention (quality decays as the window fills; context rot sets in long before the hard limit). Filling the window because it's there is spending a budget because it exists.
The discipline follows from the metaphor. Spend only on what the current task needs. Keep durable knowledge outside the window, in specs and CLAUDE.md files (Split AI coding plans into a spec and tickets). Send exploration to subagents so only conclusions come back (Orchestrator tax). Compact instead of accumulating (Agentic Context Management), and move deterministic work out of the model entirely (Use code for control flow, agents for reasoning).
One nuance: a sequence of closely related tasks (same structure, same process, same shape of work) is a legitimate reason to keep spending within one session. Once an agent has the right context in memory (i.e., conventions, examples, and corrections from earlier items), next turns benefit from the knowledge. For example, batch-creating notes that all follow the same template/structure, discuss closely related topics, etc.
But think about where the run should END. Push the batch too far and you enter the worst part of the context window, where hallucinations creep up and auto-compact is just around the corner. Better to stop a related run deliberately while quality is high than to let the budget run out mid-task. What I generally do is leverage the context window while the context window is not too crowded, and ask for a lessons learned summary that I feed into a new session, with a fresh context.
The people getting the most out of AI aren't the ones with the biggest windows; they're the ones spending tokens deliberately, and investing time into learning proper Context Engineering.
References
- OpenAI's Codex context reduction and the discussion around it: https://news.ycombinator.com/item?id=48965850
Related
About Sébastien
Ready to get to the next level?
Found this valuable? Share it with someone who needs it.