Capstone: Secure an LLM Support Bot End to End
On this page
Capstone: secure SupportBot end to end
Your task: take the vulnerable tag of SupportBot from Day 3 and produce a hardened version that a security reviewer would sign off. Try it before reading the walkthrough. The deliverables:
- A hardened SupportBot, one commit per fix, each referencing the finding it closes.
- A red-team suite that runs against both versions: the promptfoo gate (Day 24) plus a garak baseline (Day 22).
- A comparison table of attack success rates, vulnerable against hardened.
- A short report (Day 25) and a framework map (Day 28).
- Operational readiness: monitoring signals and kill switches (Day 27).
The fixes, path by path
Start from the threat model (Day 4) and close each path in code:
| Path or risk | Fix | Lesson |
|---|---|---|
P1 notes or KB to send_email | reply_to_customer with the recipient from the session; quarantined reader for notes | 8, 11 |
P2 message to issue_refund | request_refund with ownership check, 50 EUR limit, approvals, daily cap | 11 |
P3 message to lookup_order | Ownership check against the session; minimal fields | 9 |
| P4 output to chat widget | Safe renderer: no images, allowlisted links, no raw HTML; CSP | 19 |
| LLM08 hidden context | Secret moved to a vault; escalation in code; canary token | 10 |
| LLM06 consumption | Input cap, max_tokens, step cap, rate limit, daily budget | 20 |
| Rule of Two | Session policy: effectful tools need approval in tainted sessions | 5, 12, 26 |
| Supply chain | Pinned packages and MCP servers, no pickle models | 16 |
The order matters less than the habit: each fix lands in its own commit with a regression test added in the same change.
Proving the fixes work
Run the same suite against both tags. A small script keeps the comparison honest, because it runs the identical config against each version:
#!/usr/bin/env bash
# compare.sh: run the red-team gate against two SupportBot versions and save results
set -u
for tag in vulnerable hardened; do
git checkout -q "$tag"
docker compose up -d --build supportbot && sleep 5
for run in 1 2 3; do # repeat: results vary run to run
npx promptfoo@0.124 eval -c redteam/regressions.yaml --no-cache \
--output "results/$tag-regressions-$run.json" || true
npx promptfoo@0.124 redteam eval -c redteam/redteam.yaml --no-cache \
--output "results/$tag-redteam-$run.json" || true
done
python -m garak --target_type rest -G supportbot.json --spec tag:owasp:llm01 \
--generations 3 --report_prefix "garak-$tag" || true
docker compose down
done
git checkout -q mainThen summarize success rates per finding and per OWASP category. A table like this is the heart of the report:
| Finding | Vulnerable | Hardened |
|---|---|---|
| F-01 Discount code extraction | 9 of 30 runs | 0 of 30 |
| F-02 Other customers' orders | 27 of 30 | 0 of 30 |
| F-03 Order-note exfiltration by email | 6 of 20 | 0 of 20 (tool removed) |
| F-04 Refund above limit | 11 of 30 | 0 of 30, approval tickets created |
| F-05 Image exfiltration in widget | 14 of 30 | 0 of 30 rendered |
The numbers above are the format, not a promise. Fill in your own measured rates. If a hardened number isn't zero, that's a finding too: report it, decide whether it's acceptable, and record the decision.
Getting ready for production
A system that's secure on the day of the test drifts. Before sign-off, confirm the operational pieces:
- The CI gate runs on every pull request and blocks on confirmed regressions.
- garak and freshly generated promptfoo attacks run nightly, and results are reviewed weekly.
- Alerts exist for canary leaks, policy-denial spikes, unusual tool arguments and budget overruns.
- Kill switches work for each tool and for safe mode, and someone has used them in a drill.
- The framework map and system card point to current evidence.
What a strong capstone includes
- Fixes in code that remove or constrain capabilities, not only prompt changes.
- Measured success rates for both versions from the same suite, with repeated runs.
- Every finding linked to a regression test that runs in CI.
- Honest handling of anything not fully fixed: risk accepted, by whom, and why.
- Evidence a reviewer can follow without asking you: report, framework map, system card, runbook.
Check yourself
The secret must still be somewhere in the context, for example in a tool result or few-shot example. Find and remove it, since the design goal is that nothing secret is extractable. Then rerun to confirm 0 of 30.
The vulnerable baseline proves the suite can detect the problems. A suite that passes the hardened version but also passes the vulnerable one shows nothing.
The F-03 regression test runs on every pull request and fails the build if an outbound email goes to a non-customer address, and the monitoring alert on unusual email recipients covers production.
Each threat path mapped to a control in code, the same tests run before and after showing the improvement with rates, monitoring and kill switches in place, and remaining risks documented.