Few-shot prompting

What examples actually do, mechanically

An example shows the model what a finished answer looks like, and the underlying ability that makes this work is called in-context learning: a model adjusting its behaviour within a single call, from examples placed in the prompt, without any weight update at all. The leading research intuition for why this works involves what are informally called induction-like circuits inside the network — patterns learned during training that let the model notice “this input matched that pattern before, with this output” and extrapolate the same relationship to a new input later in the same context. Whatever the exact mechanism, the practical consequence is what matters here: examples are read as a pattern to continue, not as instructions to reason about, which is exactly why they are so effective for form and comparatively weak for judgement. Showing three correctly classified borderline cases does not convey the rule used to classify them; it conveys three points, and the model infers a boundary from those points that may not be the boundary you had in mind.

Zero-shot, one-shot, few-shot, many-shot

Zero-shot means no examples at all, relying entirely on the instruction. One-shot provides a single example, often enough to fix an output format. Few-shot, the usual case, provides a handful — typically somewhere between two and ten. Many-shot, made practical by larger context windows, provides dozens or hundreds, and research has found that performance on some tasks keeps improving well past the point where few-shot gains flatten out, though at a steep cost in tokens per call. The right point on this spectrum is a trade-off between cost and task difficulty: a simple classification task may need no examples at all, while a task with a genuinely unusual, hard-to-state output convention can benefit from many.

Choosing and ordering examples

A set of examples should cover the real range of inputs, including the awkward ones — the edge case, the ambiguous input, the one where the correct answer is “not enough information to decide.” A set made only of clean, easy cases teaches the model that clean, easy cases are all that exist, and it will then force a confident answer onto a genuinely ambiguous input rather than flagging it, because nothing in the examples showed that option.

Order matters, and not in a stable, predictable way: models are measurably sensitive to the sequence examples appear in, with the final example typically carrying disproportionate weight on what comes next. If your examples happen to be ordered by something incidental — every positive case first, say, purely because that is how they were pulled from a spreadsheet — the model may read that incidental ordering as part of the pattern itself. Shuffling the example order and checking whether the output changes noticeably is a cheap, worthwhile test; if it does change, that instability is itself a finding worth turning into an eval case.

Dynamic few-shot, and inverse scaling

A more advanced technique retrieves the most relevant examples for each new input at runtime, from a larger bank, rather than using the same fixed handful every time — mechanically identical to the retrieval covered in similarity search, applied to examples instead of documents. This tends to outperform a static set once the input distribution is varied enough that no fixed handful covers it well.

A genuinely counterintuitive finding worth knowing: on some tasks, adding few-shot examples has been shown to hurt the strongest models, a pattern researchers call inverse scaling. A sufficiently capable model sometimes follows a clear instruction correctly on its own, and the examples end up anchoring it toward an incidental pattern in the examples rather than the instruction’s actual intent. The practical takeaway is not to avoid examples, but to test with and without them on your own eval set rather than assuming more guidance is always better.