How to Align an LLM Judge with Human Labels (TPR, TNR)
On this page
How do you know an LLM judge is right?
Yesterday's groundedness judge reads well. That proves nothing. Until you compare its verdicts with a human expert's on the same outputs, you don't know whether a 92 percent pass rate means DocsBot is good or the judge is lenient.
Alignment means measuring that agreement and improving the judge prompt until it's good enough to trust. It's the step most teams skip, and it's the one that makes judge scores worth reporting.
Step 1: label outputs by hand
Take DocsBot outputs from your dataset runs and have the domain expert label each one pass or fail on this one criterion, with a one-line reason. You need both outcomes well represented: aim for at least a few dozen failures. If failures are rare, add outputs from a weaker prompt or older version to get enough of them.
Around 100 to 200 labeled examples is a practical start for one judge.
Step 2: split the human labels
You're about to tune the judge prompt, so treat the labels like training data:
| Split | Share | Use |
|---|---|---|
| Train | about 10 to 20% | Source of the few-shot examples in the judge prompt |
| Dev | about 40% | Measure the judge, read disagreements, revise the prompt |
| Test | about 40% | Measure once at the end, for the numbers you report |
If you put dev examples into the prompt, or keep revising until test looks good, your agreement numbers stop meaning anything.
Step 3: measure judge agreement with TPR and TNR, not accuracy
Suppose 90 percent of DocsBot answers are grounded. A judge that passes everything is 90 percent accurate and completely useless. So measure the two kinds of agreement separately:
- True positive rate (TPR): of the outputs the human passed, the share the judge also passed.
- True negative rate (TNR): of the outputs the human failed, the share the judge also failed.
Here "positive" means pass. A low TPR means the judge is too strict. A low TNR means it lets bad answers through, which is usually worse.
# alignment.py: compare judge verdicts with human labels on the dev split
def agreement(pairs: list[tuple[str, str]]) -> dict:
"""pairs: (human_label, judge_label), each 'pass' or 'fail'."""
tp = sum(h == "pass" and j == "pass" for h, j in pairs)
fn = sum(h == "pass" and j == "fail" for h, j in pairs)
tn = sum(h == "fail" and j == "fail" for h, j in pairs)
fp = sum(h == "fail" and j == "pass" for h, j in pairs)
return {"tpr": tp / (tp + fn), "tnr": tn / (tn + fp),
"accuracy": (tp + tn) / len(pairs), "n": len(pairs)}
# 80 passes and 20 failures by the human; the judge misses 8 of the failures
dev = [("pass", "pass")] * 77 + [("pass", "fail")] * 3 + [("fail", "fail")] * 12 + [("fail", "pass")] * 8
print({k: round(v, 2) for k, v in agreement(dev).items()}){'tpr': 0.96, 'tnr': 0.6, 'accuracy': 0.89, 'n': 100}Accuracy of 89 percent hides the problem: the judge catches only 60 percent of ungrounded answers.
Step 4: read the disagreements, then revise the prompt
Look at every dev example where the judge and the human disagree, especially the false passes. The critiques usually reveal why:
- The expert applies a rule the prompt doesn't state, such as "a price from the wrong currency page is unsupported".
- The judge treats a reasonable paraphrase as unsupported. Add an example showing it's fine.
- The human label is wrong. It happens. Fix the label and note it.
Revise the prompt, rerun on dev, repeat. When TPR and TNR are both high enough for your use, often above 90 percent, run once on test and record those numbers with the judge version.
Keeping the judge aligned over time
A judge prompt is code: version it, and re-check alignment when you change the judge's model, the prompt, or when the app's outputs change shape. A new DocsBot feature can produce answers the judge has never seen. A small fresh sample of human labels each month catches drift early.
Check yourself
Not without TPR and TNR. If only 5 percent of outputs fail, a judge that passes everything gets 95 percent accuracy while catching no failures at all. TNR shows how many real failures it catches.
Then you'd be measuring the judge on examples it has already seen, which inflates agreement. Few-shot examples come from the train split only.
It passes 40 percent of answers the expert failed. Your eval will report DocsBot as better than it is, and regressions in that failure mode can slip through.
TPR is the share of outputs humans passed that the judge also passed. TNR is the share of outputs humans failed that the judge also failed. You need both to be high.
Label a fresh sample regularly, especially after model or prompt changes, and recheck TPR and TNR on it. Judges drift when their model changes or the outputs change.