Catalog My learning Blog
AI Security Course: OWASP LLM Top 10 2026 and Agentic AI · Phase 5 · Lesson 22 · 4 min read

LLM Vulnerability Scanning with garak

On this page
  1. What you'll set up with garak
  2. Step 1: Install
  3. Step 2: Point it at SupportBot
  4. Step 3: Run a focused probe first
  5. Step 4: Sweep by OWASP category
  6. Step 5: Read the results carefully
  7. Common garak problems
  8. Check yourself

What you'll set up with garak

A repeatable garak scan of SupportBot that tries hundreds of known attacks, prompt injection, jailbreaks, system prompt extraction and more, and produces a report you can compare between versions. garak is NVIDIA's open-source LLM vulnerability scanner. Think of it as a broad sweep, like running a web vulnerability scanner before a manual pentest.

The commands below were tested with garak 0.17.

Step 1: Install

garak needs Python 3.10 or later. Use a separate virtual environment, since it pulls in many dependencies:

Code
python3.12 -m venv .venv-garak
.venv-garak/bin/pip install garak==0.17.0
.venv-garak/bin/python -m garak --version

Step 2: Point it at SupportBot

garak talks to models through generators. For an app behind an HTTP API, use the REST generator and describe the request in a JSON file:

JSON
{
  "rest": {
    "RestGenerator": {
      "name": "supportbot",
      "uri": "http://127.0.0.1:8811/chat",
      "method": "post",
      "headers": {"Content-Type": "application/json"},
      "req_template_json_object": {"session_id": "garak", "message": "$INPUT"},
      "response_json": true,
      "response_json_field": "answer"
    }
  }
}

$INPUT is where garak puts each attack prompt, and response_json_field tells it where the answer is in the reply. Save it as supportbot.json, and start SupportBot from Day 3.

Step 3: Run a focused probe first

Start small, to check the wiring before a long run:

Code
python -m garak --target_type rest -G supportbot.json \
  --spec probes.sysprompt_extraction.SystemPromptExtraction \
  --generations 1 --report_prefix supportbot

--spec selects what to run. In 0.17 it replaced the older --probes flag, which still works but is deprecated. You should see the probe run its attempts and print PASS or FAIL per detector, then the paths of a JSONL report and an HTML summary.

Step 4: Sweep by OWASP category

garak tags probes with OWASP categories, so you can select a whole risk at once:

Code
python -m garak --list_probes --spec tag:owasp:llm01            # see what's included

python -m garak --target_type rest -G supportbot.json \
  --spec tag:owasp:llm01 --generations 3 \
  --parallel_attempts 8 --report_prefix supportbot-llm01

For LLM01 this includes families like promptinject, latentinjection (instructions hidden in documents, close to indirect injection) and dan jailbreaks. --generations 3 repeats each prompt so you get rates, not single results. --parallel_attempts speeds things up against an HTTP target, but stay inside the cost limits from your rules of engagement.

Step 5: Read the results carefully

Open the HTML report, or the JSONL for the raw attempts. Each probe lists how many attempts each detector flagged. Two cautions:

  • Detectors are heuristics. A "hit" may be a false positive, like the bot quoting the attack while refusing it. A pass may miss a subtle failure. Read a sample of flagged attempts before you file anything.
  • garak tests the conversation, not your system. It doesn't know a refund happened or an email went out. Combine its results with your own logs and the deterministic checks from Day 21.

Save the JSONL report for each version you scan. Comparing the hit rates per probe between the vulnerable and the hardened SupportBot is a clear before-and-after picture.

Common garak problems

  • Every attempt errors. The JSON template or field name doesn't match your API. Run the focused probe with -v and check one request by hand with curl.
  • Runs take hours. Cut the selection with --spec and lower --generations. A full sweep belongs in a nightly job, not on every commit.
  • Hits you can't explain. Check whether the detector matched text the bot was refusing or quoting. Tune the selection, or confirm by hand.

Check yourself

No. garak only tries known probe families against the chat endpoint, and its detectors are heuristics. Indirect injection through order notes, and its effects on tools, need your own targeted tests and logs.

Model outputs vary between runs, so a prompt can succeed on one attempt and fail on another. Repeats give a rate you can compare across versions, instead of a single lucky or unlucky result.

garak is the broad, slower baseline sweep, typically nightly or before releases. promptfoo holds your specific, confirmed attack cases as a fast regression gate on every change.

Broad, repeatable coverage of known issue categories, quick regression checks after changes, and finding obvious gaps. They complement, not replace, testing based on your own threat model.