AI Engineer
Building products on top of models you did not train. Thirty-six lessons from what a token is through retrieval, agents, evaluation and safety, to the work of keeping an LLM feature alive in production.
What is an AI engineer?How LLMs workTokens and tokenizationEmbeddingsThe context windowCalling a model APISampling parametersStructured outputChoosing a modelPrompt anatomyFew-shot promptingChain of thoughtContext engineeringPrompt cachingChunkingVector databasesSimilarity searchThe RAG pipelineRAG or fine-tuningWhat is an agent?Tool callingThe agent loopModel Context ProtocolMulti-agent systemsWhy evals matterBuilding an eval setLLM as judgeTracing and observabilityPrompt injectionGuardrails and moderationBias and fairnessPrivacy and data handlingFine-tuningMultimodal AICost and latencyShipping and monitoring
What you are actually building on
- What is an AI engineer?A discipline that builds products on models it did not train, sitting between the ML engineer who trains them and the developer who consumes an API.
- How LLMs workNext-token prediction, a learned function with billions of parameters, and why the output is a probability distribution rather than a lookup.
- Tokens and tokenizationSubword tokenization, byte-pair encoding, and why token counts rather than word counts are the unit of both cost and context.
- EmbeddingsDense vectors whose geometry encodes meaning, where they come from, and what they are and are not good for.
Talking to a model
- The context windowThe window as a budget you spend rather than a setting you configure, what happens at the limit, and why a bigger one is not a strategy.
- Calling a model APIMessages and roles, the request and response envelope, tokens in and out, and the error handling every production call needs.
- Sampling parametersTemperature, top-k, top-p and repetition penalties — what each one does to the distribution, and when touching it is justified.
- Structured outputGetting schema-conforming JSON out of a model reliably, using constrained decoding rather than asking politely and hoping.
- Choosing a modelOpen weights against closed APIs, size against latency and cost, and why a benchmark leaderboard is the weakest evidence available to you.
Prompt and context engineering
- Prompt anatomyThe parts of a working prompt — role, task, context, constraints, output format — and why structure beats the search for magic words.
- Few-shot promptingZero-shot against few-shot, how to choose and order examples, and the cases where examples make the output worse.
- Chain of thoughtGetting a model to work before it answers, from step-by-step prompting to reasoning models that do it internally.
- Context engineeringDeciding what goes in the window and why — write, select, compress, isolate — and the failure modes of a window packed to the limit.
- Prompt cachingHow a cached prefix removes repeated prefill work, how to arrange a prompt so the cache actually hits, and what it saves.
Giving the model your data
- ChunkingSplitting documents for retrieval, and why chunk boundaries decide retrieval quality more than the embedding model does.
- Vector databasesWhat a vector index actually does, how HNSW and IVF trade accuracy for speed, and when Postgres with pgvector is enough.
- Similarity searchScoring, top-k, hybrid search with BM25, and re-ranking — why vector search alone underperforms on real queries.
- The RAG pipelineIngest, chunk, embed, index, retrieve, re-rank, generate — the whole path, with the failure mode waiting at each stage.
- RAG or fine-tuningWhich problem each one actually solves, why fine-tune it on our docs is usually the wrong instinct, and when the answer is neither.
Models that take actions
- What is an agent?A model using tools in a loop to reach a goal — where that genuinely helps, and where a fixed pipeline is the better engineering.
- Tool callingDeclaring tools, executing what the model asks for, feeding results back — and designing tools a model can actually use correctly.
- The agent loopPlan, act, observe, repeat — owning the control loop, deciding when to stop, and why agents stall or run away.
- Model Context ProtocolThe problem MCP solves, how hosts, clients and servers fit together, and what a standard interface changes about wiring an agent to the world.
- Multi-agent systemsOrchestrators and sub-agents, handoffs, isolated context — and the honest case that one good agent usually beats five.
Knowing whether it works
- Why evals matterWhy iterating on vibes stops working, what an eval actually is, and the error-analysis loop that turns failures into measurements.
- Building an eval setSourcing real failures, labelling them, writing assertions that do not need a model, and keeping the set from going stale.
- LLM as judgeGrading output with a model, aligning that judge against human labels, and the biases that make an unvalidated judge worse than none.
- Tracing and observabilityWhy LLM traces are a different problem from uptime monitoring, what a span should carry, and finding the failing step in an agent run.
Where it goes wrong
- Prompt injectionWhy instructions and data share one channel, direct and indirect injection, the lethal trifecta, and why no complete fix exists.
- Guardrails and moderationInput and output checks, classifiers and moderation APIs, human review — and an honest account of what guardrails can promise.
- Bias and fairnessWhere bias enters a model and a product built on one, how it is measured, and what an application engineer can actually do about it.
- Privacy and data handlingHow user data moves through prompts, logs and retrieval, what provider terms actually say, and designing a system that does not leak by default.
Shipping it
- Fine-tuningFull fine-tuning against LoRA, what adaptation teaches and what it does not, and the real cost of owning a tuned model.
- Multimodal AIHow images, audio and video reach a language model, what changes in an application when the input is not text, and where the costs move.
- Cost and latencyToken economics, time to first token, caching, routing and batching — the levers that decide whether an LLM feature is affordable.
- Shipping and monitoringRollout and fallbacks, provider outages and rate limits, versioning prompts and models, and watching quality after launch.