AI Code Review: Bots vs Human Reviewers
On this page
Learning Objective
Add automated AI review to your pull request process in a way that helps rather than adds noise: decide what AI review should check, configure it with your team's criteria, measure its precision, and keep humans responsible for design and approval.
Concept Explanation
As agents increase the volume of code, review capacity becomes the constraint (Day 14). AI review tools offer a first pass on every pull request: GitHub Copilot code review, Claude Code's Code Review and its review workflow for GitHub Actions, CodeRabbit, and others. Used well, they catch real bugs before a human looks, let reviewers spend their attention on design and intent, and give authors feedback within minutes. Used badly, they post dozens of trivial or wrong comments that people learn to ignore, which is worse than nothing.
Decide what the bot is for. AI review is strongest at finding concrete defects in the diff: logic errors, unhandled edge cases, missing null checks, injection and authorization gaps, inconsistencies between code and tests, and violations of written team conventions. It is weaker at judging whether the change is the right change, whether the design fits the system's direction, and whether the trade-offs suit the business. Those remain human responsibilities, and the approval that merges code should remain a human one.
Give the reviewer your criteria. A generic review produces generic comments. Point the bot at the same conventions your agents follow, the instruction file and any review checklist, and state what to ignore: formatting and style that linters already enforce, nits that do not affect correctness, and speculative refactors. Ask for findings ordered by severity with file, line and a concrete fix, and for silence when nothing significant is found. A review that says "no significant issues" is a valuable signal, not a failure.
Measure signal-to-noise. For a few weeks, have authors react to each bot comment as useful or not, or track how many comments lead to a code change. If fewer than about half of comments are acted on, tighten the instructions or reduce the scope. Track also what the bot misses by comparing with bugs found later. Review bots, like any tool, deserve tuning based on evidence.
There is one more benefit: the reviewer is a different model or at least a different context from the agent that wrote the code (Day 18). Independent review catches mistakes the author agent is blind to, much as a second human reviewer does. Some teams run the review in the agent's own loop before the pull request is opened, so obvious problems never reach humans at all.
| Review concern | Who is better placed | Note |
|---|---|---|
| Concrete defects in the diff | AI first pass, human confirms | Logic, edge cases, error handling |
| Security gaps | AI plus static analysis | Injection, authorization, secrets |
| Team conventions | AI with written criteria | Only conventions that are written down |
| Style and formatting | Linters and formatters | Exclude from AI review entirely |
| Design and direction | Human | Fit with architecture and roadmap |
| Approval to merge | Human | Accountability stays with people |
Worked Example: A Tuned Review Workflow
This workflow runs the code-review plugin on every pull request and posts inline comments. A section in the instruction file sets the review criteria that the bot and the team's agents share.
# .github/workflows/review.yml
name: Code Review
on:
pull_request:
types: [opened, synchronize, ready_for_review, reopened]
jobs:
review:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 1
- uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
plugin_marketplaces: "https://github.com/anthropics/claude-code.git"
plugins: "code-review@claude-code-plugins"
prompt: "/code-review:code-review --comment ${{ github.repository }}/pull/${{ github.event.pull_request.number }}"
claude_args: '--allowedTools "mcp__github_inline_comment__create_inline_comment"'<!-- AGENTS.md, review section -->
## Review criteria (for humans, agents and the review bot)
- Report: correctness bugs, unhandled errors, missing authorization via policy.can(),
queries not using the repository layer, tests that were weakened or removed,
money handled as floats, personal data in logs.
- Ignore: formatting and lint-enforced style, naming preferences, optional refactors.
- Each finding: severity, file:line, the concrete problem, a suggested fix.
- If nothing significant is found, say so in one line.The workflow keeps permissions read-only except for posting comments through the allowed tool. Because the criteria live in the shared instruction file, the agent that writes code and the bot that reviews it apply the same rules, and human reviewers can focus on whether the change is the right one.
Real-World Examples
- Claude Code's documentation describes both a managed Code Review product that reviews every pull request automatically and a code-review plugin that can run in GitHub Actions, posting inline comments for issues found or a single summary comment when it finds none.
- GitHub Copilot code review became generally available in 2025 and can be requested as a reviewer on pull requests or enabled automatically for a repository, with custom instructions files used to steer what it checks.
Review Questions & Answers
Concrete defects visible in the diff, such as logic errors, unhandled edge cases, missing error handling, security gaps like injection or missing authorization, test inconsistencies and violations of written conventions.
Linters and formatters enforce them deterministically. Including them in AI review creates noisy comments that train people to ignore the bot.
Measure signal-to-noise, for example the share of comments that authors mark useful or that lead to code changes, and compare against bugs found later that the bot missed.
Accountability for what ships, and judgment about design, direction and trade-offs, belong to people. AI review reduces their load but does not carry responsibility.