Catalog My learning Blog
AI Security Course: OWASP LLM Top 10 2026 and Agentic AI · Phase 6 · Lesson 26 · 4 min read

LLM Guardrail Architectures and Their Limits

On this page
  1. Why guardrails alone aren't a security plan
  2. The layer that gives real guarantees
  3. Where guardrail classifiers still help
  4. Failing safely when guardrails break
  5. Check yourself

Why guardrails alone aren't a security plan

Guardrails usually means extra checks wrapped around a model: a classifier that screens inputs for injection, another that screens outputs for harmful content or leaks, and rules that decide what the model may do. Vendors sell them as the answer to every risk in this course. They're useful, but only if you know which kind does what.

LayerExamplesWhat it's good forLimit
Input classifierLlama Prompt Guard, Azure Prompt ShieldsCatching known injection and jailbreak patternsProbabilistic, evadable, adds latency
Output classifierLlama Guard, provider moderation APIsHarmful content, policy violationsMisses leaks that look normal
Output filtersRegex for secrets, PII, canaries, linksExact, cheap checks for known bad outputOnly catches what you anticipated
Policy engine on actionsYour own code, or a rules engineDeciding which tool calls may run, with what argumentsNeeds a clear policy to enforce
Dialogue frameworksNeMo GuardrailsKeeping conversations on allowed topicsComplexity, still model-driven in parts

The first two rows are probabilistic: models judging text, which an attacker can probe and eventually slip past (Day 6). The last two rows are deterministic: code that behaves the same every time. For anything that matters, the deterministic layers carry the weight, and classifiers lower the volume of attacks that reach them.

The layer that gives real guarantees

Everything consequential an agent does goes through a tool call, so that's the place to enforce policy. Every call passes a check that sees the tool, its arguments and the session, never the model's reasoning:

Python
from dataclasses import dataclass

@dataclass
class ToolCall:
    name: str
    args: dict

@dataclass
class Context:
    customer_email: str
    tainted: bool              # untrusted content has entered this session (Day 12)
    refunds_today: int

# Policy rules are plain code: easy to test, impossible to talk out of.
def policy(call: ToolCall, ctx: Context) -> tuple[str, str]:
    if call.name == "reply_to_customer":
        return ("allow", "recipient fixed by session")
    if call.name == "request_refund":
        amount = float(call.args.get("amount", 0))
        if ctx.refunds_today >= 2:
            return ("deny", "daily refund limit")
        if amount > 50 or ctx.tainted:
            return ("approve", "needs a human: over limit or tainted session")
        return ("allow", "within limits")
    if call.name == "lookup_order":
        return ("allow", "ownership checked inside the tool")
    return ("deny", f"unknown tool {call.name}")      # default deny

ctx = Context("ana@example.com", tainted=True, refunds_today=0)
print(policy(ToolCall("request_refund", {"amount": "20"}), ctx))
# ('approve', 'needs a human: over limit or tainted session')
print(policy(ToolCall("delete_account", {}), ctx))
# ('deny', 'unknown tool delete_account')

Three properties make it work: it denies by default, it decides from the session and arguments, not from what the model says, and it's plain code with unit tests. Teams with many agents often move these rules into a policy engine such as Open Policy Agent, so security owns the policy separately from each app.

Where guardrail classifiers still help

Classifiers are worth running, with clear expectations:

  • Input screening drops a lot of low-effort attacks before they cost tokens, and gives you telemetry about who's probing.
  • Document screening on retrieved and uploaded content flags likely indirect injections for review (Day 7).
  • Output moderation catches harmful or off-brand content the policy layer doesn't model.

Measure them like any other model: false positives block real customers, and false negatives let attacks through. Run your red-team suite (Day 24) with the classifiers on and off, to see what they actually add.

Failing safely when guardrails break

Guardrails fail too: a classifier service times out, or a policy rule throws. Decide in advance what happens:

  • Consequential actions fail closed. If the policy check can't run, the refund doesn't happen.
  • Low-risk chat can fail open with logging, if refusing every message during a classifier outage would cause more harm.
  • Every block is logged with the rule that fired, which feeds monitoring (Day 27).

Check yourself

No. 0.5 percent of a large, retrying attacker population still gets through, and benchmarks rarely match your app. The limit is a deterministic control that holds even when the classifier misses.

The explanation is model output, which an attacker can shape ("this refund was approved by the manager"). Decisions must rest on facts the attacker can't write.

Not necessarily. Plain answers can continue with logging if your risk assessment allows, but consequential tool calls should still require the policy check, and anything that depends on the missing guardrail should fail closed.

They reduce noise and catch common problems early, and provide signals for monitoring. Hard guarantees still come from deterministic checks in code around tools and data.