LlamaIndex for agents

A framework built from the data outward

Most agent frameworks start from the agent loop and treat data access as one tool among many. LlamaIndex started from the opposite end: its first and still central job is turning a pile of unstructured documents — PDFs, wikis, codebases, support tickets — into an index that a query can search efficiently, then handing the retrieved chunks to an LLM to answer a question. Everything downstream, including its current agent layer, is built on top of that indexing and query machinery rather than bolted onto a framework that was agent-first from day one. This is a real architectural difference, not just branding: the retrieval layer in LlamaIndex is the oldest, most developed part of the project, and its agent abstractions are designed to call into that layer naturally.

What this buys an agent that leans on retrieval

Consider an internal agent meant to answer employee questions by searching a company’s policy documents, past incident reports, and engineering wikis — tens of thousands of pages, updated constantly, with answers that genuinely depend on finding the right three paragraphs out of that pile. Building this well means solving problems that have nothing to do with the agent loop itself: how do you chunk a hundred-page PDF so that relevant context doesn’t get split across chunk boundaries, how do you combine results from differently-structured sources, how do you re-rank candidates before they hit the model’s context window, how do you keep the index current as documents change. LlamaIndex has spent years accumulating answers to exactly these problems — multiple chunking strategies, a range of index types beyond flat vector search, query engines that can route a question to the right sub-index. An agent built with LlamaIndex’s AgentWorkflow inherits all of that for free, which is a genuine head start over wiring retrieval together by hand inside a more general-purpose orchestration framework.

Where that strength stops mattering

Now consider a different agent: one that mostly calls a handful of internal APIs, checks their results, and decides what to do next, with little or no document retrieval involved. Here, LlamaIndex’s deepest investment — its indexing and query layer — sits mostly unused. The agent still needs the things every framework needs to provide: a loop, state, tool-calling conventions, maybe branching logic. A framework built agent-first, with orchestration as its central concern rather than a later addition, is often the more natural fit for this kind of agent, because its core abstractions were designed around exactly that problem rather than growing out to cover it.

Retrieval-first versus orchestration-first

This is the real axis to compare frameworks on, and it’s worth being honest about rather than treating every framework as a universal, interchangeable choice. LangGraph and similar frameworks are orchestration-first: they have a clear, general opinion about control flow — nodes, edges, shared state — and little opinion about how you should store or retrieve your data, leaving that to you or to a library you plug in. LlamaIndex is retrieval-first: it has a clear, mature opinion about indexing and querying data, and its agent orchestration, while genuinely capable, is the newer part of the stack built to make use of that strength. Neither is a strictly better design; they’re optimized for different centers of gravity in what the agent actually spends its time doing.

Choosing with this in mind

A reasonable rule: if describing your agent’s job mostly involves “search across a large collection of our own documents and answer based on what’s found,” LlamaIndex’s retrieval-first design is doing real, load-bearing work for you, and its agent layer is a sensible default. If the agent’s job is mostly “call these tools, check results, decide what’s next,” with retrieval as a minor piece if present at all, an orchestration-first framework is likely to feel more direct, because you won’t be carrying the weight of a retrieval system you aren’t really using. Many real systems end up combining both — a LlamaIndex-built retrieval tool called from an agent loop built in something else — which is itself evidence that these two strengths are genuinely complementary rather than competing claims to the same job.