LLM Misinformation and Unbounded Consumption
On this page
Misinformation: wrong answers are your answers
In February 2024 a Canadian tribunal ordered Air Canada to honor a bereavement refund policy its website chatbot had made up. The airline argued the chatbot was responsible for its own words. The tribunal disagreed: the company is responsible for everything on its website, chatbot included. In 2023, lawyers in Mata v. Avianca were sanctioned for filing a brief citing court cases ChatGPT had invented.
LLM07, Misinformation, rose two places in 2026. It covers confident, plausible answers that are wrong, from hallucinated facts to invented policies, citations and package names. Attackers can make it worse with injection or poisoning, but it happens without them too.
Controls, in order of strength:
- Don't let the model decide facts that matter. Refund terms, prices and eligibility come from your systems through tools, and the answer quotes them.
- Ground answers in retrieved sources and tell the model to say "I don't know" when the sources don't cover the question.
- Check grounding before showing high-stakes answers, for example by verifying that every number and policy statement appears in the retrieved text. The LLM Evals course covers faithfulness checks in depth.
- Show sources so people can verify, and label the assistant as AI.
- Escalate to a human for legal, medical, financial or account-changing questions.
For SupportBot, that means the refund window in an answer must come from the policy tool's output, not from the model's memory of "typical" policies.
Unbounded consumption: cost as an attack
LLM06, Unbounded Consumption, jumped four places. Every token costs money and compute, so anyone who can make your app use a lot of them can hurt you. That might be a deliberate "denial of wallet" attack, someone using your bot as a free general-purpose model, or a bug like an agent stuck in a loop.
| Abuse | Example | Limit |
|---|---|---|
| Huge inputs | Pasting a whole book into the chat | Maximum input size |
| Huge outputs | "Repeat this word 100,000 times" | max_tokens on every call |
| Runaway agents | A tool error loops forever | Maximum tool steps per request |
| Request floods | A script hitting the endpoint | Rate limits per user and IP |
| Freeloading | Using your bot as a free general model | Daily token budgets per user, topic limits |
| Model extraction | Mass queries to copy your fine-tuned model | Rate limits, monitoring for scraping patterns |
Rate and cost caps at the front door
import time
from dataclasses import dataclass, field
MAX_INPUT_CHARS = 4_000
MAX_OUTPUT_TOKENS = 1_024 # pass as max_tokens on every model call
MAX_TOOL_STEPS = 6 # the agent loop stops after this many tool rounds
DAILY_TOKEN_BUDGET = 200_000 # per customer
@dataclass
class Budget:
used: dict[str, int] = field(default_factory=dict)
day: str = ""
def charge(self, user: str, tokens: int) -> None:
today = time.strftime("%Y-%m-%d")
if today != self.day:
self.day, self.used = today, {}
self.used[user] = self.used.get(user, 0) + tokens
def remaining(self, user: str) -> int:
return DAILY_TOKEN_BUDGET - self.used.get(user, 0)
class RateLimit:
"""Token bucket: `rate` requests per second, bursts up to `burst`."""
def __init__(self, rate: float, burst: int):
self.rate, self.burst, self.buckets = rate, burst, {}
def allow(self, key: str, now: float | None = None) -> bool:
now = time.monotonic() if now is None else now
tokens, last = self.buckets.get(key, (self.burst, now))
tokens = min(self.burst, tokens + (now - last) * self.rate)
if tokens < 1:
self.buckets[key] = (tokens, now)
return False
self.buckets[key] = (tokens - 1, now)
return True
def admit(user: str, message: str, budget: Budget, limiter: RateLimit, now=None) -> str | None:
"""Return None to proceed, or a refusal message."""
if len(message) > MAX_INPUT_CHARS:
return "Message too long."
if not limiter.allow(user, now):
return "Too many requests. Please wait a moment."
if budget.remaining(user) <= 0:
return "Daily usage limit reached."
return None
b, rl = Budget(), RateLimit(rate=0.5, burst=3)
print([admit("ana", "hi", b, rl, now=0.0) for _ in range(4)])
# [None, None, None, 'Too many requests. Please wait a moment.']
b.charge("ana", 200_000) # after each call, charge usage.input_tokens + output_tokens
print(admit("ana", "hi", b, rl, now=10.0))
# Daily usage limit reached.After each model call, charge the user with the real token counts from the response's usage field. Keep the budget in shared storage such as Redis in production, so every server sees the same numbers. And set a hard monthly spend limit with your model provider, plus an alert well below it. That's the backstop if everything else fails.
Check yourself
Your company is, as the Air Canada ruling showed. Policy facts should come from a tool that returns the actual policy, and answers should quote it rather than rely on the model's memory.
A cap on tool steps per request, plus max_tokens on each call and a per-user budget. Any one of them ends the loop; together they bound the cost.
Real usage includes system prompts, tool definitions, retrieved context and outputs, which can dwarf the user's message. Only the reported counts reflect what you pay.
Ground answers in retrieved, current policy documents with citations, evaluate accuracy on policy questions, and route binding decisions like refunds to systems that apply the real rules.