LLM security testing tools — garak, Promptfoo, and PyRIT

Automation needs software, and the field has converged on a few

The previous lesson in this group made the case that automated testing matters because it gives you breadth and repeatability that manual testing cannot. That case only holds up if the tooling actually exists and is good. It does — AI red teaming has produced a small set of open-source tools that most practitioners reach for, each with a distinct niche. Knowing which one fits a given job is itself a practical skill, and it’s the subject of this lesson.

garak: broad vulnerability scanning

garak, maintained by NVIDIA, is built around the same basic idea as a conventional vulnerability scanner, just retargeted at language models. It organizes known attack and failure categories into “probes” — prompt injection, jailbreak patterns, data leakage, toxic output generation, and more — runs a target model through each one, and uses a “detector” to judge whether the model’s response counts as a failure for that category. The output is a report that tells you, across a wide spread of known issue categories, where a given model currently stands. This makes garak a strong default for the first question any engagement asks: before doing anything custom, what does this model already fail at that is already well understood in the field? Because its probe library is broad rather than deep on any one category, garak is better suited to a wide first pass than to exhaustively stress-testing one specific weakness.

Promptfoo: red teaming that lives in CI

Promptfoo takes a different angle, aimed squarely at developers building LLM applications who want security testing to behave like the rest of their test suite. It’s a CLI tool that can run both evaluation (does the model give good answers) and red-teaming (does the model resist known attacks) as part of a normal development workflow — install it, point it at your application’s prompts and configuration, run a scan, and get a report that’s designed to be read by a developer, not just a security specialist. Its real value is in where it sits: because it’s built to be CI-integrated, a team can catch a regression — a prompt change that reopens an old jailbreak, or a new feature that introduces a fresh injection surface — automatically on every pull request, the same way a team already catches a broken unit test. That makes Promptfoo the natural fit when the goal is ongoing, low-friction testing baked into how an application is normally built and shipped, rather than a standalone security engagement.

PyRIT: a framework for building your own attacks

PyRIT, Microsoft’s open-source Python Risk Identification Toolkit, is not a fixed scanner with a built-in probe library in the way garak is — it’s an orchestration framework that gives you composable building blocks: prompt converters (for encoding or rephrasing an attack), attack strategies (single-turn, multi-turn, iterative refinement), targets (the model or system under test), and scoring logic for judging success. The tradeoff for that flexibility is that PyRIT expects you to assemble a strategy rather than run a single command and get a report. This makes it the right tool reach for when the previous two don’t fit the job — when a red teamer wants to compose a custom, multi-step automated attack, such as an adaptive jailbreak that uses one model to iteratively refine prompts against another, or a strategy chaining several of the techniques covered earlier in this roadmap together in a way no off-the-shelf probe library anticipated.

Picking a starting point

A reasonable rule of thumb: reach for garak first when you want a broad read on known weaknesses with minimal setup, reach for Promptfoo when the goal is wiring red-teaming into a real development pipeline so regressions get caught automatically, and reach for PyRIT when the attack you actually need doesn’t exist yet in a prebuilt library and has to be composed from primitives. None of the three replaces manual testing’s capacity to find something genuinely new — they automate exactly the known-pattern sweep the previous lesson described, at different points in a team’s workflow.