Catalog My learning Blog
AI Security Course: OWASP LLM Top 10 2026 and Agentic AI · Phase 6 · Lesson 30 · 3 min read

Capstone: Secure an LLM Support Bot End to End

On this page
  1. Capstone: secure SupportBot end to end
  2. The fixes, path by path
  3. Proving the fixes work
  4. Getting ready for production
  5. What a strong capstone includes
  6. Check yourself

Capstone: secure SupportBot end to end

Your task: take the vulnerable tag of SupportBot from Day 3 and produce a hardened version that a security reviewer would sign off. Try it before reading the walkthrough. The deliverables:

  1. A hardened SupportBot, one commit per fix, each referencing the finding it closes.
  2. A red-team suite that runs against both versions: the promptfoo gate (Day 24) plus a garak baseline (Day 22).
  3. A comparison table of attack success rates, vulnerable against hardened.
  4. A short report (Day 25) and a framework map (Day 28).
  5. Operational readiness: monitoring signals and kill switches (Day 27).

The fixes, path by path

Start from the threat model (Day 4) and close each path in code:

Path or riskFixLesson
P1 notes or KB to send_emailreply_to_customer with the recipient from the session; quarantined reader for notes8, 11
P2 message to issue_refundrequest_refund with ownership check, 50 EUR limit, approvals, daily cap11
P3 message to lookup_orderOwnership check against the session; minimal fields9
P4 output to chat widgetSafe renderer: no images, allowlisted links, no raw HTML; CSP19
LLM08 hidden contextSecret moved to a vault; escalation in code; canary token10
LLM06 consumptionInput cap, max_tokens, step cap, rate limit, daily budget20
Rule of TwoSession policy: effectful tools need approval in tainted sessions5, 12, 26
Supply chainPinned packages and MCP servers, no pickle models16

The order matters less than the habit: each fix lands in its own commit with a regression test added in the same change.

Proving the fixes work

Run the same suite against both tags. A small script keeps the comparison honest, because it runs the identical config against each version:

Code
#!/usr/bin/env bash
# compare.sh: run the red-team gate against two SupportBot versions and save results
set -u
for tag in vulnerable hardened; do
  git checkout -q "$tag"
  docker compose up -d --build supportbot && sleep 5
  for run in 1 2 3; do                                  # repeat: results vary run to run
    npx promptfoo@0.124 eval -c redteam/regressions.yaml --no-cache \
      --output "results/$tag-regressions-$run.json" || true
    npx promptfoo@0.124 redteam eval -c redteam/redteam.yaml --no-cache \
      --output "results/$tag-redteam-$run.json" || true
  done
  python -m garak --target_type rest -G supportbot.json --spec tag:owasp:llm01 \
    --generations 3 --report_prefix "garak-$tag" || true
  docker compose down
done
git checkout -q main

Then summarize success rates per finding and per OWASP category. A table like this is the heart of the report:

FindingVulnerableHardened
F-01 Discount code extraction9 of 30 runs0 of 30
F-02 Other customers' orders27 of 300 of 30
F-03 Order-note exfiltration by email6 of 200 of 20 (tool removed)
F-04 Refund above limit11 of 300 of 30, approval tickets created
F-05 Image exfiltration in widget14 of 300 of 30 rendered

The numbers above are the format, not a promise. Fill in your own measured rates. If a hardened number isn't zero, that's a finding too: report it, decide whether it's acceptable, and record the decision.

Getting ready for production

A system that's secure on the day of the test drifts. Before sign-off, confirm the operational pieces:

  • The CI gate runs on every pull request and blocks on confirmed regressions.
  • garak and freshly generated promptfoo attacks run nightly, and results are reviewed weekly.
  • Alerts exist for canary leaks, policy-denial spikes, unusual tool arguments and budget overruns.
  • Kill switches work for each tool and for safe mode, and someone has used them in a drill.
  • The framework map and system card point to current evidence.

What a strong capstone includes

  • Fixes in code that remove or constrain capabilities, not only prompt changes.
  • Measured success rates for both versions from the same suite, with repeated runs.
  • Every finding linked to a regression test that runs in CI.
  • Honest handling of anything not fully fixed: risk accepted, by whom, and why.
  • Evidence a reviewer can follow without asking you: report, framework map, system card, runbook.

Check yourself

The secret must still be somewhere in the context, for example in a tool result or few-shot example. Find and remove it, since the design goal is that nothing secret is extractable. Then rerun to confirm 0 of 30.

The vulnerable baseline proves the suite can detect the problems. A suite that passes the hardened version but also passes the vulnerable one shows nothing.

The F-03 regression test runs on every pull request and fails the build if an outbound email goes to a non-customer address, and the monitoring alert on unusual email recipients covers production.

Each threat path mapped to a control in code, the same tests run before and after showing the improvement with rates, monitoring and kill switches in place, and remaining risks documented.