Catalog My learning Blog
AI-Assisted Coding Course: Agentic Development in 30 Days · Phase 3 · Lesson 12 · 5 min read

Debugging with AI: Root Causes and git bisect

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: Letting the Agent Bisect a Regression
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Debug systematically with an agent: provide reproduction steps and evidence, have the agent form and test hypotheses rather than guess fixes, use instrumentation and tools such as git bisect, and require a root-cause explanation before accepting a fix.

Concept Explanation

Agents are fast at proposing fixes, which is both their strength and their danger in debugging. Asked to "fix the error", an agent will often change the line in the stack trace until the error disappears. Sometimes that is the fix; often it moves the problem elsewhere, hides it behind a try-except, or breaks a case nobody tested. Good debugging with an agent applies the same discipline as good debugging without one: reproduce, observe, hypothesize, test, and only then fix.

Start with evidence. Give the agent the exact error message and stack trace, logs around the failure, the input that triggers it, what changed recently, and what you have already ruled out. Agents read logs and traces quickly and well, often spotting the relevant line in thousands faster than a person. If you can reproduce the bug, ask for a failing test that captures it before anything else (Day 11).

Then ask for hypotheses, not fixes. "List the three most likely causes, and for each, what evidence would confirm or rule it out" turns the agent from a patch generator into an investigator. It can then gather that evidence: add temporary logging, run the failing case with extra output, query the database, inspect the state at the failing point, or run a debugger. Agents equipped to run programs and inspect their output debug far better than those reasoning from code alone.

When a bug is a regression, git bisect is a powerful tool to hand an agent. Given a known good commit, a known bad one and a script that tests for the bug, git bisect run performs a binary search through the history and names the commit that introduced the problem. Reading that commit's diff usually makes the cause obvious, and the agent can explain why the change broke the behavior.

Require a root-cause explanation before accepting a fix. The explanation should say what was wrong, why the symptom appeared, why the fix addresses the cause rather than the symptom, and whether the same mistake exists elsewhere. If the agent cannot explain why its change works, treat the fix as a guess. Finally, keep the reproducing test as a regression test, and remove temporary logging before committing.

StepAsk the agent toAvoid
ReproduceWrite a failing test or script that shows the bugFixing a bug nobody reproduced
ObserveRead logs, traces, add temporary outputGuessing from the stack trace line alone
HypothesizeList likely causes and the evidence for eachJumping to the first plausible patch
LocateUse git bisect for regressionsReading the whole history by hand
Fix and explainFix the root cause, explain why it workstry-except that hides the error
ProtectKeep the test as a regression testLeaving debug logging behind

Worked Example: Letting the Agent Bisect a Regression

The agent writes a small script that exits non-zero when the bug is present, then runs git bisect between the last good release and the current main branch.

Code
# check_bug.sh: exit 0 if behavior is correct, 1 if the bug is present, 125 to skip a commit.
#!/usr/bin/env bash
set -u
pnpm install --frozen-lockfile --silent >/dev/null 2>&1 || exit 125   # unbuildable commit: skip
out=$(pnpm --silent tsx scripts/repro-login-plus.ts 2>&1) || true
if grep -q "LOGIN_OK" <<<"$out"; then exit 0; else exit 1; fi
Shell
git bisect start
git bisect bad main            # bug present now
git bisect good v4.12.0        # last release known to work
git bisect run ./check_bug.sh  # binary search: about 7 steps for 100 commits
# -> 3f9c2ab is the first bad commit: "feat: parse form bodies in shared middleware"
git bisect reset
git show 3f9c2ab -- src/middleware/body.ts

With the offending commit in hand, the agent can explain the root cause precisely: the new middleware parses JSON login bodies as URL-encoded forms, turning plus signs into spaces. Exit code 125 tells bisect to skip commits that do not build, which keeps the search reliable across messy history.

Give the agent evidence, have it reproduce the bug with a failing test, list hypotheses with the evidence that would confirm each, gather that evidence or bisect the history, then fix the root cause and explain it.
Give the agent evidence, have it reproduce the bug with a failing test, list hypotheses with the evidence that would confirm each, gather that evidence or bisect the history, then fix the root cause and explain it.

Real-World Examples

  • Microsoft Research's debug-gym project (2025) gave language-model agents interactive debugging tools such as pdb and reported that agents performed better on debugging benchmarks when they could set breakpoints and inspect state rather than only read code.
  • git bisect run has been part of Git since 2007 and is widely used in projects such as the Linux kernel to find regressions automatically, which makes it a natural tool to give an agent that can write the test script.
Key TakeawaysDebug with a coding agent the way a careful engineer debugs: give it evidence and reproduce the bug with a failing test first. Ask for hypotheses and the evidence that would confirm them, let it instrument and run code, and use git bisect for regressions. Accept a fix only with a root-cause explanation, keep the reproducing test, and remove temporary instrumentation.

Review Questions & Answers

The agent may change code until the error disappears without understanding the cause, for example by suppressing the exception or moving the failure elsewhere, which hides the bug instead of fixing it.

It shifts the agent from generating patches to investigating: it gathers evidence such as logs, outputs and state to confirm or rule out causes before changing code.

Given a good and a bad commit and a script that returns success or failure, it performs a binary search through the commits between them, running the script on each candidate until it identifies the first bad commit.

What was wrong, why it caused the observed symptom, why the change fixes the cause rather than the symptom, and whether the same mistake exists elsewhere.