Context engineering

From wording to contents

Prompt engineering asks how to phrase an instruction. Context engineering asks what should be in the window at all — which documents, how much history, which tool definitions, which prior results, and what gets dropped to make room. For anything long-running, the second question dominates: the wording of the instruction is a small fraction of what the model is actually reading. The shift happened because systems got longer-lived. A single-turn call is a prompt. An agent working for twenty minutes across fifty tool calls is a context-management problem with a prompt somewhere inside it.

The four moves, each with concrete technique

Write means persisting state somewhere other than the conversation window — a scratchpad file the agent can append to, a running task list, a small key-value memory store — so that information survives without permanently occupying context. A coding agent that writes its plan to a file on disk, then re-reads only the relevant section later, is using this move.

Select means retrieving only what the current step actually needs, rather than carrying everything that might ever be relevant. This is retrieval applied to the agent’s own working memory, not just to an external document store: choosing which three of fifty available tools to expose for this particular step, or pulling back only the one earlier message that is actually relevant to the question just asked, is selection.

Compress means summarising history rather than carrying it verbatim — replacing the full text of the last fifteen tool calls with a three-sentence summary of what was tried and what was learned, once that detail stops earning its place in the budget. Compression is lossy by design, which makes deciding what to lose a real engineering decision rather than an automatic default: summarising away the specific error message that caused a retry can make the agent repeat the same mistake, because the detail that would have prevented it was exactly what got compressed away.

Isolate means giving a sub-task its own clean context window so its intermediate mess never pollutes the orchestrating agent’s context — the mechanism behind multi-agent systems, where a sub-agent can spend an entire window exploring a dead end and report back only its conclusion.

Context rot, with the specific finding

Research measuring model performance as input length grows — referenced in this lesson’s deeper link — finds that accuracy degrades non-uniformly: it is not a smooth, gentle slope but a pattern where certain kinds of added content, particularly irrelevant or distracting material placed among the relevant facts, hurt disproportionately more than an equal volume of simply longer but relevant text. A document that duplicates information already present, or a retrieved passage that is topically related but does not actually answer the question, can measurably drag down accuracy on the real answer sitting nearby in the same window — which is a direct argument for the re-ranking step in similarity search: a few precisely relevant passages reliably beat many loosely relevant ones.

When to compact, as a concrete heuristic

A workable rule many production agents use: trigger compaction once the context has filled to somewhere around seventy percent of the window, rather than waiting until it is full. Waiting until the last moment means compaction has to be aggressive and rushed, discarding detail indiscriminately; starting earlier allows a more careful summary that preserves what later steps are likely to need. Re-stating the original goal near the end of the context on every turn, even after heavy compaction, is a small, cheap habit that keeps a long-running agent from drifting away from what it was actually asked to do — the “losing the plot” failure covered in the agent loop.

The discipline is mostly subtractive

The question worth asking repeatedly is not what else the model might find useful, but what is currently in the window that is not earning its tokens. Nearly every context-engineering failure in production is an excess, not a deficiency — too much retrieved, too much history carried, too many tools offered at once — and the fix is almost always removal rather than addition.