Memory Poisoning and Inter-Agent Communication
On this page
Attacks that outlive the session
Most attacks so far end when the conversation ends. Memory changes that. Agents now remember preferences, facts from past conversations, summaries of earlier tasks and cached documents. ASI06, Memory and Context Poisoning, is about attackers writing malicious content into those stores so it shapes every future session.
In 2024 Johann Rehberger showed this against ChatGPT with an attack he called SpAIware. A prompt injection made it store a false memory containing an instruction to exfiltrate all future conversations. It persisted until the user found and deleted it, and OpenAI then hardened its memory handling. Shared memory is worse: one poisoned entry in a team knowledge store reaches every user.
Guarding memory writes
Treat a memory write like any privileged action:
- Record provenance. Did it come from the user explicitly, from an inference about a conversation, or from content the agent read?
- Refuse untrusted sources. Text from documents, web pages and tool results should never become a memory, or at most become clearly labeled data, never an instruction.
- Restrict the shape. Store short facts and preferences, not imperative instructions or URLs.
- Show and expire. Let users see and delete their memories, confirm changes that alter behavior, and expire old entries.
- Review shared stores. Content goes through review before it's visible to other users.
import re, time
IMPERATIVE = re.compile(r"(?i)\b(always|never|from now on|ignore|send|forward|email|visit|http)\b")
def remember(store, user_id: str, text: str, source: str) -> str:
"""source: 'user_explicit', 'conversation_inference' or 'external_content'."""
if source == "external_content":
return "Not stored: memories cannot come from documents, web pages or tool results."
if IMPERATIVE.search(text) or len(text) > 300:
return "Not stored: memories must be short facts or preferences, not instructions."
store.add(user_id=user_id, text=text, source=source, created=time.time(), ttl_days=180)
return "Saved. You can review or delete memories in Settings."The keyword check is crude on purpose. The important rule is the first one: provenance decides, so a perfectly phrased instruction from a web page still can't get in.
Attacks that spread between agents
ASI07, Insecure Inter-Agent Communication, covers systems where agents talk to each other: an orchestrator handing work to workers, or agents from different teams connected over protocols like A2A or MCP. Messages between agents are a new attack surface. They can be forged if agents don't authenticate senders, replayed or tampered with in transit, and they can carry injections: a worker that was manipulated returns a result containing instructions for the orchestrator, and the attack spreads.
The controls come straight from service-to-service security. Every agent gets its own identity, with mutual TLS or signed tokens. Messages are signed and carry a timestamp and a nonce. Here's the idea with a shared HMAC key per agent:
import hashlib, hmac, json, time, uuid
KEYS = {"billing-agent": b"per-agent secret from a vault"}
SEEN_NONCES: set[str] = set()
def sign(sender: str, payload: dict) -> dict:
msg = {"sender": sender, "ts": int(time.time()), "nonce": uuid.uuid4().hex, "payload": payload}
body = json.dumps(msg, sort_keys=True).encode()
msg["sig"] = hmac.new(KEYS[sender], body, hashlib.sha256).hexdigest()
return msg
def verify(msg: dict, max_age_s: int = 60) -> dict:
msg = dict(msg)
sig = msg.pop("sig", "")
body = json.dumps(msg, sort_keys=True).encode()
expected = hmac.new(KEYS[msg["sender"]], body, hashlib.sha256).hexdigest()
if not hmac.compare_digest(sig, expected):
raise PermissionError("bad signature")
if abs(time.time() - msg["ts"]) > max_age_s or msg["nonce"] in SEEN_NONCES:
raise PermissionError("stale or replayed message")
SEEN_NONCES.add(msg["nonce"])
return msg["payload"] # still untrusted content: validate its schema before use
print(verify(sign("billing-agent", {"invoice": "7781", "status": "paid"})))In production, mTLS or tokens from your identity provider replace the shared keys, and the nonce store is shared and expiring. The principle stays: an authenticated message is still only data. Authentication proves who sent it, not that its content is safe, so validate it against a schema and keep it away from privileged decisions, exactly like a document.
| Store or channel | Attack | Control |
|---|---|---|
| Personal memory | Injected instruction saved as a memory | Provenance, no imperative memories, user review |
| Shared memory | A poisoned entry reaches all users | Review before publishing, source tracking |
| Agent messages | Forged sender, replay, tampering | Agent identities, signatures, timestamps, nonces |
| Worker results | An injection propagates to the orchestrator | Schema validation, treat as untrusted |
Check yourself
Text from a page, possibly attacker-written, becomes a persistent memory that shapes every future session. Store only what the user typed or explicitly confirmed, as a short fact, never page content as instructions.
Treat the content as untrusted data. The valid signature only proves which agent sent it, and that agent may have been manipulated. Validate the fields against the expected schema and never act on instructions inside them.
A signature proves an old message is genuine, so an attacker could resend it. The nonce, plus the timestamp check, lets the receiver reject replays.
Only store memories the user explicitly confirmed, record their source, keep them as data rather than instructions, and let users review and delete them.