Refactoring Legacy Code with AI: Characterization Tests
On this page
Learning Objective
Use agents to change code you did not write and do not fully understand: build characterization tests first, refactor in small behavior-preserving steps, decide between agent edits and deterministic codemods for large migrations, and keep every step reviewable.
Concept Explanation
Legacy code is where agents can save the most time and do the most damage. They read unfamiliar code tirelessly, explain it, and propose modernizations quickly. But legacy code is often legacy precisely because nobody knows all of what it does, and an agent's confident rewrite can silently drop behavior that some customer depends on. The discipline that makes refactoring safe for people makes it safe for agents: lock in current behavior, then change structure in small steps.
Start with understanding. Ask the agent to explain the module: its responsibilities, inputs and outputs, side effects, callers, and anything surprising. Ask it to list the implicit behaviors it can see, such as special cases for certain customers, error swallowing, or reliance on ordering. This explanation is cheap and often reveals risks before any change.
Next, write characterization tests. Unlike tests written from a specification, characterization tests record what the code does today, even if that behavior looks wrong. Michael Feathers introduced the term in Working Effectively with Legacy Code. Agents are good at generating them: feed a range of representative and edge-case inputs through the existing code and assert on the outputs, or capture outputs as approved snapshots. These tests turn "do not change behavior" from a hope into a check.
Then refactor in small steps, each one behavior-preserving and verified: extract a function, rename for clarity, remove duplication, introduce a seam for testing, replace a deprecated API. Ask the agent for one step at a time and run the characterization tests after each. Behavior changes, including bug fixes you discovered along the way, are separate commits with their own tests, so reviewers can tell refactoring from functional change.
Large migrations, such as moving hundreds of files from one test library or framework version to another, need a strategy. When the transformation is mechanical and uniform, a deterministic codemod built with tools such as jscodeshift, ast-grep or OpenRewrite is more reliable and reviewable than many agent edits, and agents are good at writing codemods. When each file needs judgment, run the agent file by file or in small batches, with tests per batch and a tracking list of what is done. Many successful migrations combine both: a codemod for the mechanical bulk, an agent for the files the codemod cannot handle.
| Technique | Purpose | Agent's role |
|---|---|---|
| Explain the module | Surface responsibilities and hidden behaviors | Read and summarize, list risks |
| Characterization tests | Lock in current behavior before changes | Generate inputs and record outputs |
| Small refactoring steps | Change structure without changing behavior | One step at a time, tests after each |
| Separate behavior changes | Keep fixes reviewable | Separate commits with new tests |
| Codemods | Uniform mechanical migrations | Write and run the transformation |
| Batched agent migration | Changes needing per-file judgment | File-by-file edits with tracking |
Worked Example: Characterization Tests Before Touching a Legacy Function
The agent is asked to write tests that pin down the current behavior of a tangled shipping-cost function, including the odd parts, before any refactoring.
# tests/test_shipping_characterization.py
# Characterization tests: they record what calculate_shipping DOES today, not what it should do.
import json, pathlib, pytest
from legacy.shipping import calculate_shipping
SNAPSHOT = pathlib.Path(__file__).with_name("shipping_snapshot.json")
CASES = [
{"weight_kg": 0.5, "country": "US", "express": False, "customer_tier": "standard"},
{"weight_kg": 0.5, "country": "US", "express": True, "customer_tier": "standard"},
{"weight_kg": 20, "country": "CA", "express": False, "customer_tier": "gold"},
{"weight_kg": 0, "country": "US", "express": False, "customer_tier": "standard"}, # zero weight
{"weight_kg": 31, "country": "DE", "express": True, "customer_tier": "gold"}, # over limit
{"weight_kg": 2, "country": "XX", "express": False, "customer_tier": "standard"}, # unknown country
]
def run(case):
try:
return {"ok": str(calculate_shipping(**case))}
except Exception as exc: # errors are behavior too
return {"error": type(exc).__name__}
def test_matches_recorded_behavior():
actual = [run(c) for c in CASES]
if not SNAPSHOT.exists(): # first run records the baseline; review it
SNAPSHOT.write_text(json.dumps(actual, indent=2))
pytest.skip("Snapshot recorded; review shipping_snapshot.json and commit it.")
assert actual == json.loads(SNAPSHOT.read_text())The first run records the baseline, which you review: if unknown countries silently cost zero, that is a bug worth knowing about, but the refactoring must preserve it until a separate, explicit fix changes it. From then on, every refactoring step must keep the snapshot identical.
Real-World Examples
- Airbnb reported in 2025 that it migrated about 3,500 React test files from Enzyme to React Testing Library with an LLM-driven pipeline in six weeks, work it had estimated at a year and a half by hand, using automated validation and retries per file.
- Google described in 2025 how it uses LLMs for large internal code migrations, combining AI-generated changes with automated verification and human review, and reported that a large share of the changes in those migrations were AI-authored while total migration time dropped substantially.
Review Questions & Answers
Characterization tests record what the code currently does, even if it seems wrong, so changes can be checked for unintended behavior differences. Specification tests describe what the code should do.
Reviewers can verify that refactoring commits change structure only, with tests unchanged, and can evaluate behavior changes on their own merits with their own tests.
When the transformation is uniform and mechanical across many files. A deterministic codemod applies it consistently and is easier to review than many independently generated edits.
Raising an exception for certain inputs is part of the current behavior. A refactor that turned an error into a silent default, or the reverse, would change behavior and should be caught.