Catalog My learning Blog
LLM Evals Course: Error Analysis, LLM Judges and RAG Evals · Phase 2 · Lesson 8 · 5 min read

LLM-as-a-Judge: How to Write a Pass/Fail Judge Prompt

On this page
  1. What is LLM-as-a-judge?
  2. Four rules for writing judge prompts
  3. Example: a groundedness judge prompt
  4. Things that quietly break LLM judges
  5. Where LLM judges fit in an eval suite
  6. Check yourself

What is LLM-as-a-judge?

"Is every claim in this answer supported by the retrieved pages?" No regex can answer that. A person can, but reading 300 answers per prompt change doesn't scale. An LLM judge is a model call that reads the input, the output and any reference material, and returns a verdict on one specific criterion.

Judges are powerful and easy to get wrong. A judge that sounds reasonable but disagrees with your domain expert a third of the time produces confident, wrong numbers. This lesson covers writing one well. Day 9 covers proving it's good.

Four rules for writing judge prompts

  1. One failure mode per judge. "Rate the overall quality" mixes accuracy, tone and completeness into a number nobody can act on. Write one judge for "unsupported claim", another for "unneeded handoff".
  2. Binary verdicts, not 1 to 5 scales. People and models disagree about what separates a 3 from a 4, and averages of scale scores are hard to interpret. Pass or fail forces a clear definition. If you need nuance, split the criterion into two binary checks.
  3. Critique before verdict. Ask the judge to explain its reasoning first, then decide. The critique also tells you why it failed something when you review disagreements.
  4. Give it what a human grader would need. For groundedness, that's the retrieved pages. Without them the judge grades against its own knowledge of Acme, which it doesn't have.

Example: a groundedness judge prompt

Python
# judge_grounded.py: does every factual claim in the answer appear in the retrieved docs?
import anthropic
from typing import Literal
from pydantic import BaseModel

client = anthropic.Anthropic()

JUDGE_PROMPT = """You are grading answers from Acme's documentation assistant.

Criterion: every factual claim about Acme's product in the ANSWER must be supported by the
DOCUMENTS. Supported means stated in, or directly implied by, the documents.

PASS if all product claims are supported, or the answer makes no product claims (for example
a handoff to a human). Phrasing differences are fine.
FAIL if any claim about features, limits, prices, plans or steps is missing from or
contradicts the documents. One unsupported claim is enough to fail.

Examples:
- Docs say reset links last 1 hour; answer says 24 hours. FAIL: contradicts the docs.
- Docs describe CSV export; answer also says "you can export to Excel". FAIL: Excel not in docs.
- Answer: "I'll connect you with our support team." PASS: no product claims.

Write a short critique that names each product claim and whether it is supported, then
give the verdict."""

class Verdict(BaseModel):
    critique: str
    verdict: Literal["pass", "fail"]

def judge_grounded(question: str, documents: list[str], answer: str) -> Verdict:
    docs = "\n\n".join(f"<doc>{d}</doc>" for d in documents)
    response = client.messages.parse(
        model="claude-opus-5-5", max_tokens=1000,
        system=JUDGE_PROMPT,
        messages=[{"role": "user", "content":
            f"<question>{question}</question>\n<documents>{docs}</documents>\n<answer>{answer}</answer>"}],
        output_format=Verdict,
    )
    return response.parsed_output

The field order in Verdict puts the critique first, so the model writes its reasoning before committing to a verdict. The examples in the prompt come from real disagreements in your error analysis, which is where the hard cases live.

Things that quietly break LLM judges

  • Asking the judge to grade its own output. Studies of LLM judges, such as the MT-Bench work by Zheng and colleagues in 2023, found that models can favor their own answers. When you can, use a different model as the judge than the one under test, and always validate against humans.
  • Hidden criteria. If your domain expert fails answers for being "too salesy" but the prompt never says so, the judge won't either. Write down every rule the expert applies.
  • Missing context. A judge for "respects plan" needs the customer's plan. Pass it in.
  • Vague words. "Accurate", "helpful" and "appropriate" mean different things to different readers. Replace them with concrete conditions, as the prompt above does for "supported".

Where LLM judges fit in an eval suite

Judges cost money and time, so run them where code can't decide. A typical DocsBot run checks citations and plan rules with code, then runs the groundedness judge on answers that make product claims. On 300 examples that's 300 extra model calls, which you'll weigh on Day 29.

Check yourself

Split it into binary pass or fail judges, one per failure mode, each with a concrete definition and examples of passes and failures.

Writing the reasoning first tends to produce more careful verdicts, and the critique shows you why it decided, which you need when it disagrees with a human.

Without the documents the judge can only use its own knowledge, which doesn't include Acme's docs. Groundedness judges must receive the retrieved text.

A judge asked about several things at once mixes them into a vague verdict and is hard to align. Separate judges for groundedness, tone and completeness each give a clear signal.

They show the judge where the line is in concrete cases. Use examples that aren't in your dev or test sets, so alignment measurements stay honest.