Catalog My learning Blog
AI Security Course: OWASP LLM Top 10 2026 and Agentic AI · Phase 3 · Lesson 15 · 4 min read

Memory Poisoning and Inter-Agent Communication

On this page
  1. Attacks that outlive the session
  2. Guarding memory writes
  3. Attacks that spread between agents
  4. Check yourself

Attacks that outlive the session

Most attacks so far end when the conversation ends. Memory changes that. Agents now remember preferences, facts from past conversations, summaries of earlier tasks and cached documents. ASI06, Memory and Context Poisoning, is about attackers writing malicious content into those stores so it shapes every future session.

In 2024 Johann Rehberger showed this against ChatGPT with an attack he called SpAIware. A prompt injection made it store a false memory containing an instruction to exfiltrate all future conversations. It persisted until the user found and deleted it, and OpenAI then hardened its memory handling. Shared memory is worse: one poisoned entry in a team knowledge store reaches every user.

Guarding memory writes

Treat a memory write like any privileged action:

  • Record provenance. Did it come from the user explicitly, from an inference about a conversation, or from content the agent read?
  • Refuse untrusted sources. Text from documents, web pages and tool results should never become a memory, or at most become clearly labeled data, never an instruction.
  • Restrict the shape. Store short facts and preferences, not imperative instructions or URLs.
  • Show and expire. Let users see and delete their memories, confirm changes that alter behavior, and expire old entries.
  • Review shared stores. Content goes through review before it's visible to other users.
Python
import re, time

IMPERATIVE = re.compile(r"(?i)\b(always|never|from now on|ignore|send|forward|email|visit|http)\b")

def remember(store, user_id: str, text: str, source: str) -> str:
    """source: 'user_explicit', 'conversation_inference' or 'external_content'."""
    if source == "external_content":
        return "Not stored: memories cannot come from documents, web pages or tool results."
    if IMPERATIVE.search(text) or len(text) > 300:
        return "Not stored: memories must be short facts or preferences, not instructions."
    store.add(user_id=user_id, text=text, source=source, created=time.time(), ttl_days=180)
    return "Saved. You can review or delete memories in Settings."

The keyword check is crude on purpose. The important rule is the first one: provenance decides, so a perfectly phrased instruction from a web page still can't get in.

Attacks that spread between agents

ASI07, Insecure Inter-Agent Communication, covers systems where agents talk to each other: an orchestrator handing work to workers, or agents from different teams connected over protocols like A2A or MCP. Messages between agents are a new attack surface. They can be forged if agents don't authenticate senders, replayed or tampered with in transit, and they can carry injections: a worker that was manipulated returns a result containing instructions for the orchestrator, and the attack spreads.

The controls come straight from service-to-service security. Every agent gets its own identity, with mutual TLS or signed tokens. Messages are signed and carry a timestamp and a nonce. Here's the idea with a shared HMAC key per agent:

Python
import hashlib, hmac, json, time, uuid

KEYS = {"billing-agent": b"per-agent secret from a vault"}
SEEN_NONCES: set[str] = set()

def sign(sender: str, payload: dict) -> dict:
    msg = {"sender": sender, "ts": int(time.time()), "nonce": uuid.uuid4().hex, "payload": payload}
    body = json.dumps(msg, sort_keys=True).encode()
    msg["sig"] = hmac.new(KEYS[sender], body, hashlib.sha256).hexdigest()
    return msg

def verify(msg: dict, max_age_s: int = 60) -> dict:
    msg = dict(msg)
    sig = msg.pop("sig", "")
    body = json.dumps(msg, sort_keys=True).encode()
    expected = hmac.new(KEYS[msg["sender"]], body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(sig, expected):
        raise PermissionError("bad signature")
    if abs(time.time() - msg["ts"]) > max_age_s or msg["nonce"] in SEEN_NONCES:
        raise PermissionError("stale or replayed message")
    SEEN_NONCES.add(msg["nonce"])
    return msg["payload"]          # still untrusted content: validate its schema before use

print(verify(sign("billing-agent", {"invoice": "7781", "status": "paid"})))

In production, mTLS or tokens from your identity provider replace the shared keys, and the nonce store is shared and expiring. The principle stays: an authenticated message is still only data. Authentication proves who sent it, not that its content is safe, so validate it against a schema and keep it away from privileged decisions, exactly like a document.

Store or channelAttackControl
Personal memoryInjected instruction saved as a memoryProvenance, no imperative memories, user review
Shared memoryA poisoned entry reaches all usersReview before publishing, source tracking
Agent messagesForged sender, replay, tamperingAgent identities, signatures, timestamps, nonces
Worker resultsAn injection propagates to the orchestratorSchema validation, treat as untrusted

Check yourself

Text from a page, possibly attacker-written, becomes a persistent memory that shapes every future session. Store only what the user typed or explicitly confirmed, as a short fact, never page content as instructions.

Treat the content as untrusted data. The valid signature only proves which agent sent it, and that agent may have been manipulated. Validate the fields against the expected schema and never act on instructions inside them.

A signature proves an old message is genuine, so an attacker could resend it. The nonce, plus the timestamp check, lets the receiver reject replays.

Only store memories the user explicitly confirmed, record their source, keep them as data rather than instructions, and let users review and delete them.