The agent loop

Four lines, then the hard part

Call the model with the conversation and the available tools. If it asks for a tool, run it. Append the result to the conversation. Repeat until it answers directly or you stop it. That is the entire loop, and writing it takes minutes. Everything difficult lives in the gaps between those four lines: what exactly goes into the context on iteration twelve, what happens when a tool fails twice in a row, how you detect that the agent is alternating uselessly between the same two calls, and when you decide it has had enough attempts. None of that is in the loop itself; all of it determines whether the agent actually works in practice.

Memory: short-term context versus long-term storage

An agent’s short-term memory is simply whatever is currently in its context window — the conversation so far, including every tool call and result from the current run. This disappears the moment the run ends unless explicitly saved. Long-term memory is state deliberately persisted outside the context window — a file, a database row, a note — that can be read back in on a future run or even by a different agent entirely. The context engineering lesson’s “write” move is exactly this: moving information from the temporary short-term space into something durable before it would otherwise be lost to compaction or to the run simply ending.

Explicit planning versus step-by-step improvisation

Some agent designs ask the model to produce an explicit plan up front — a numbered list of sub-tasks — before executing any of them, then work through that plan largely as written, adjusting only when a step fails outright. This tends to produce more coherent, legible traces and is easier to debug, at the cost of being slower to adapt when an early step reveals the plan itself was wrong. Other designs have the model decide one step at a time with no advance plan at all, reacting purely to what the previous step returned — more adaptive to genuine surprises, but with a real risk of local, myopic decisions that never add up to coherent overall progress. A reflection step, where the agent periodically pauses to critique its own progress against the original goal before continuing, is a common middle ground used with either design, catching drift before it compounds across many further steps.

A worked trace, narrated

Consider an agent asked to find and summarise a customer’s recent support history. Step one: it calls a lookup tool with the customer’s email and gets back a customer ID. Step two: it calls a tickets tool with that ID and gets back a list of ten ticket identifiers with no detail. Step three: realising it needs the actual ticket content, it calls a ticket-detail tool — but gets back a result for the wrong ticket, because of a subtle identifier format mismatch between the two tools. Step four: the model notices the mismatch in the returned data, reasons about what likely went wrong, and retries with a corrected identifier format. Step five: now holding the correct ten tickets’ content, it synthesises a summary and answers. Nothing about this sequence was specified in advance — the length, the detour in step three, and the self-correction in step four all emerged from the model reasoning about what each result actually contained, which is both the entire value of an agent over a fixed pipeline and exactly the kind of run that is hard to predict, test, or bound in advance.

ReAct, and what it actually added

Interleaving explicit reasoning with each action — think about what to do next, act, observe the result, think again before acting again — is usually called the ReAct pattern. Its real contribution was demonstrating that a model reasoning between actions, rather than simply emitting one tool call after another with no stated justification, noticeably outperforms the reasoning-free version, because each observation gets genuinely interpreted before the next decision rather than pattern-matched straight into the next call. Modern tool-calling APIs have largely absorbed this behaviour implicitly, which is why it is rarely implemented as an explicit separate step any more — but when an agent is making visibly poor choices, giving it more explicit room to reason between steps, rather than adding more tools, is still often the right first fix.

Why agents stall, and why they run away

Stalling looks like the same tool called repeatedly with the same or nearly the same arguments. The underlying cause is almost always that the previous result carried no genuinely new, usable signal — an empty list, an opaque error with no actionable detail — leaving the model nothing new to condition its next decision on. Detecting a repeated call and injecting something different — a hint, a narrowed scope, or simply a forced stop — is the standard fix.

Running away is the opposite failure: each step generates plausible further work, and the agent never naturally concludes it is done. This is what explicit budgets are for, and they need to cover several axes at once, because each catches a different kind of runaway: a cap on the number of iterations, a cap on total tokens consumed, a wall-clock timeout, and a hard cost ceiling. An agent that respects three of these four will eventually find the one you forgot to set.

Losing the plot comes from the window gradually filling with accumulated tool output until the original goal sits far back in the context and is weighted only weakly by attention. Compaction, as covered in context engineering, is the structural remedy, and simply re-stating the original goal near the end of the context on every iteration costs almost nothing and helps more than its simplicity suggests.

Record everything the loop does, in full. The specific run you will need to understand later is, by the nature of non-determinism, one you cannot simply reproduce by trying again — which is exactly what tracing exists to capture.