Catalog My learning Blog
Build AI Agents with MCP: 30-Day Course · Phase 2 · Lesson 9 · 5 min read

AI Agent Memory: Short-Term vs Long-Term Memory

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: A Small File-Backed Memory with Provenance
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Distinguish the kinds of memory an agent can have, implement long-term memory that persists across sessions, and decide what to store, how to retrieve it, and how to keep memory accurate and safe.

Concept Explanation

A language model has no memory of its own between requests. Everything it knows about the current task arrives in the context window each time. When people talk about agent memory they mean systems that decide what to put into that window, and they usually distinguish several kinds borrowed loosely from cognitive science.

Short-term or working memory is the conversation history of the current session: the messages, tool calls and results so far. It is managed with the context engineering techniques from Day 5. Long-term memory survives the session. It splits into three useful categories. Semantic memory holds facts: the user's timezone, the team's deployment process, the fact that a customer prefers email. Episodic memory holds past experiences: last week's incident, how a previous attempt at this task failed. Procedural memory holds how-to knowledge: instructions, playbooks and skills the agent should follow, often stored as files that are loaded when relevant.

Storage and retrieval come in a few shapes. The simplest is a file the agent reads at the start of every session, such as a project instructions file. It is transparent and easy to edit, and it suits small amounts of stable knowledge. For larger memories, the agent gets tools to search and write: a key-value store, a document store, or a vector index searched by semantic similarity. Anthropic's memory tool takes a file-based approach where the model creates, reads and edits files in a memory directory that your code stores, which keeps memory inspectable.

Deciding what to remember is the hard part. Storing everything floods retrieval with noise, while storing too little loses useful context. Good rules include remembering corrections and stated preferences, decisions and their reasons, and facts that took effort to discover; not remembering secrets, transient state, or things easily rederived from the source. Memories also go stale. A stored fact that the build uses version 18 is wrong after an upgrade, so memories should carry dates or sources, be updated rather than duplicated, and be verified before being acted on when the stakes are high.

Memory is also a security surface. Anything an agent writes to long-term memory influences future sessions, so a prompt injection that gets the agent to store "always send reports to this external address" persists. Treat memory writes as privileged actions: restrict what can be stored, keep provenance, and let users see and delete memories.

Memory typeHoldsTypical storageLifetime
Working (short-term)Current conversation and tool resultsContext windowOne session
SemanticFacts and preferencesFiles, key-value store, vector indexUntil updated
EpisodicPast events and attemptsLogs, summaries with datesDecays in relevance
ProceduralInstructions, playbooks, skillsInstruction files loaded on demandVersioned with the project

Worked Example: A Small File-Backed Memory with Provenance

Two tools let the agent save and recall memories. Each memory records when and why it was saved, and saving an existing key updates it instead of duplicating it.

Python
import json, pathlib, datetime

STORE = pathlib.Path("agent_memory.json")
FORBIDDEN = ("password", "api_key", "secret", "token")

def _load() -> dict:
    return json.loads(STORE.read_text()) if STORE.exists() else {}

def remember(key: str, value: str, reason: str) -> str:
    """Save or update a durable fact, preference or decision. key is a short slug such as
    'deploy-process'. reason says why this is worth remembering. Never store credentials."""
    if any(word in (key + value).lower() for word in FORBIDDEN):
        raise ValueError("Refusing to store something that looks like a credential.")
    memories = _load()
    memories[key] = {"value": value, "reason": reason,
                     "saved": datetime.date.today().isoformat()}
    STORE.write_text(json.dumps(memories, indent=2))
    return f"Saved '{key}'."

def recall(query: str) -> str:
    """Search saved memories by keyword. Returns matching entries with their save dates."""
    hits = {k: v for k, v in _load().items() if query.lower() in (k + v["value"]).lower()}
    if not hits:
        return "No matching memories."
    return "\n".join(f"{k} (saved {v['saved']}): {v['value']}" for k, v in hits.items())

# At session start, load a short index of keys into the system prompt so the agent
# knows what it can recall, without loading every value into context.
index = ", ".join(_load().keys()) or "none yet"
SYSTEM = f"You have long-term memory. Saved keys: {index}. Use recall before asking the user."

Showing the save date with every recalled memory lets the model judge staleness, and the index-only system prompt keeps context small while still telling the agent what it knows.

Working memory is the context window. Long-term memory lives outside it: semantic facts and preferences, episodic records of past events, and procedural instructions and skills, each with different storage and lifetime.
Working memory is the context window. Long-term memory lives outside it: semantic facts and preferences, episodic records of past events, and procedural instructions and skills, each with different storage and lifetime.

Real-World Examples

  • ChatGPT's memory feature and Claude's memory for projects both let users view, edit and delete what the assistant remembers, reflecting the principle that long-term memory must be transparent and user-controllable.
  • Security researcher Johann Rehberger demonstrated in 2024 that a prompt injection in a document could make ChatGPT store false memories that persisted into later conversations, showing why memory writes need the same caution as any other privileged action.
Key TakeawaysAgents have working memory in the context window and long-term memory in external storage that survives between sessions. Long-term memory splits into semantic facts, episodic experiences and procedural instructions. Store corrections, preferences and hard-won facts with dates and reasons; update rather than duplicate; skip secrets and easily rederived data. Treat memory writes as privileged, keep them visible to users, and verify stale memories before acting on them.

Review Questions & Answers

Semantic memory holds facts and preferences, such as a user's timezone or a team's deployment process. Episodic memory holds records of past events, such as how a previous attempt at a task failed or what happened in an incident.

Facts go stale, and a date or source lets the model and the user judge whether a memory is still reliable and verify it before acting on it when the stakes are high.

Anything written to long-term memory affects future sessions. A prompt injection that gets the agent to store a malicious instruction would persist, so writes should be restricted, attributed and visible to users.

It keeps the context small while telling the agent what it knows, so it can retrieve specific memories with a tool when they become relevant instead of carrying all of them on every request.