Catalog My learning Blog
AI Security Course: OWASP LLM Top 10 2026 and Agentic AI · Phase 4 · Lesson 20 · 4 min read

LLM Misinformation and Unbounded Consumption

On this page
  1. Misinformation: wrong answers are your answers
  2. Unbounded consumption: cost as an attack
  3. Rate and cost caps at the front door
  4. Check yourself

Misinformation: wrong answers are your answers

In February 2024 a Canadian tribunal ordered Air Canada to honor a bereavement refund policy its website chatbot had made up. The airline argued the chatbot was responsible for its own words. The tribunal disagreed: the company is responsible for everything on its website, chatbot included. In 2023, lawyers in Mata v. Avianca were sanctioned for filing a brief citing court cases ChatGPT had invented.

LLM07, Misinformation, rose two places in 2026. It covers confident, plausible answers that are wrong, from hallucinated facts to invented policies, citations and package names. Attackers can make it worse with injection or poisoning, but it happens without them too.

Controls, in order of strength:

  • Don't let the model decide facts that matter. Refund terms, prices and eligibility come from your systems through tools, and the answer quotes them.
  • Ground answers in retrieved sources and tell the model to say "I don't know" when the sources don't cover the question.
  • Check grounding before showing high-stakes answers, for example by verifying that every number and policy statement appears in the retrieved text. The LLM Evals course covers faithfulness checks in depth.
  • Show sources so people can verify, and label the assistant as AI.
  • Escalate to a human for legal, medical, financial or account-changing questions.

For SupportBot, that means the refund window in an answer must come from the policy tool's output, not from the model's memory of "typical" policies.

Unbounded consumption: cost as an attack

LLM06, Unbounded Consumption, jumped four places. Every token costs money and compute, so anyone who can make your app use a lot of them can hurt you. That might be a deliberate "denial of wallet" attack, someone using your bot as a free general-purpose model, or a bug like an agent stuck in a loop.

AbuseExampleLimit
Huge inputsPasting a whole book into the chatMaximum input size
Huge outputs"Repeat this word 100,000 times"max_tokens on every call
Runaway agentsA tool error loops foreverMaximum tool steps per request
Request floodsA script hitting the endpointRate limits per user and IP
FreeloadingUsing your bot as a free general modelDaily token budgets per user, topic limits
Model extractionMass queries to copy your fine-tuned modelRate limits, monitoring for scraping patterns

Rate and cost caps at the front door

Python
import time
from dataclasses import dataclass, field

MAX_INPUT_CHARS = 4_000
MAX_OUTPUT_TOKENS = 1_024             # pass as max_tokens on every model call
MAX_TOOL_STEPS = 6                    # the agent loop stops after this many tool rounds
DAILY_TOKEN_BUDGET = 200_000          # per customer

@dataclass
class Budget:
    used: dict[str, int] = field(default_factory=dict)
    day: str = ""

    def charge(self, user: str, tokens: int) -> None:
        today = time.strftime("%Y-%m-%d")
        if today != self.day:
            self.day, self.used = today, {}
        self.used[user] = self.used.get(user, 0) + tokens

    def remaining(self, user: str) -> int:
        return DAILY_TOKEN_BUDGET - self.used.get(user, 0)

class RateLimit:
    """Token bucket: `rate` requests per second, bursts up to `burst`."""
    def __init__(self, rate: float, burst: int):
        self.rate, self.burst, self.buckets = rate, burst, {}

    def allow(self, key: str, now: float | None = None) -> bool:
        now = time.monotonic() if now is None else now
        tokens, last = self.buckets.get(key, (self.burst, now))
        tokens = min(self.burst, tokens + (now - last) * self.rate)
        if tokens < 1:
            self.buckets[key] = (tokens, now)
            return False
        self.buckets[key] = (tokens - 1, now)
        return True

def admit(user: str, message: str, budget: Budget, limiter: RateLimit, now=None) -> str | None:
    """Return None to proceed, or a refusal message."""
    if len(message) > MAX_INPUT_CHARS:
        return "Message too long."
    if not limiter.allow(user, now):
        return "Too many requests. Please wait a moment."
    if budget.remaining(user) <= 0:
        return "Daily usage limit reached."
    return None

b, rl = Budget(), RateLimit(rate=0.5, burst=3)
print([admit("ana", "hi", b, rl, now=0.0) for _ in range(4)])
# [None, None, None, 'Too many requests. Please wait a moment.']
b.charge("ana", 200_000)                  # after each call, charge usage.input_tokens + output_tokens
print(admit("ana", "hi", b, rl, now=10.0))
# Daily usage limit reached.

After each model call, charge the user with the real token counts from the response's usage field. Keep the budget in shared storage such as Redis in production, so every server sees the same numbers. And set a hard monthly spend limit with your model provider, plus an alert well below it. That's the backstop if everything else fails.

Check yourself

Your company is, as the Air Canada ruling showed. Policy facts should come from a tool that returns the actual policy, and answers should quote it rather than rely on the model's memory.

A cap on tool steps per request, plus max_tokens on each call and a per-user budget. Any one of them ends the loop; together they bound the cost.

Real usage includes system prompts, tool definitions, retrieved context and outputs, which can dwarf the user's message. Only the reported counts reflect what you pay.

Ground answers in retrieved, current policy documents with citations, evaluate accuracy on policy questions, and route binding decisions like refunds to systems that apply the real rules.