Ground Truth and Calibration for Agent Evals

An eval score looks objective because it is a number. The difficult question is whether the grader’s decisions match the behavior your product actually considers acceptable.

That requires ground truth and calibration.

What ground truth means

Ground truth is the best supported reference label or expected result available for a case. It may come from:

• an exact fact in a trusted system;
• a deterministic policy;
• a verified tool outcome;
• expert human review;
• consensus across reviewers;
• a later confirmed real-world outcome.

Not every agent task has one indisputable answer. A better name in ambiguous domains is often reference judgment.

Labels need more than pass and fail

A useful label set can include:

• acceptable;
• failure;
• inconclusive;
• out of scope;
• missing evidence;
• reviewer disagreement.

Forcing every uncertain example into pass or fail contaminates the benchmark and hides where the criterion is unclear.

Separate criterion design from grader evaluation

The same examples used to write a rubric should not be the only examples used to report its quality.

Divide cases by project, session, or time into:

• development cases for writing and debugging the criterion;
• validation cases for choosing thresholds or prompts;
• held-out test cases for the final quality estimate.

Randomly splitting near-duplicate traces can leak the same failure pattern into every set. Group related runs before splitting.

What calibration measures

A calibrated confidence of 0.8 should correspond to being correct roughly 80 percent of the time across comparable judgments.

For actionable failure detection, track more than accuracy:

• precision among reported failures;
• false-positive rate;
• recall where labels are sufficiently complete;
• abstention rate;
• reviewer agreement;
• unresolved or invalid evidence citations;
• quality by project, failure type, and agent version.

High average accuracy can hide a grader that fails badly on one important group.

Reviewer disagreement is evidence

When reviewers disagree, do not automatically use majority vote and discard the discussion. Record:

• each reviewer’s label;
• the criterion version they used;
• the reason for disagreement;
• the adjudicated result, if any;
• whether the rubric changed afterward.

Disagreement can reveal unclear product requirements rather than poor reviewers.

Calibrating an LLM judge

Evaluate a judge on held-out, human-reviewed cases. Test:

• repeated calls with the same input;
• changed answer order for pairwise judging;
• irrelevant verbosity;
• adversarial text inside retrieved content or tool output;
• missing and contradictory evidence;
• a newer judge model before switching production versions.

Keep the previous judge results. A model upgrade should create new provenance, not rewrite history.

Calibrating deterministic detectors

Deterministic does not mean correct. A stable rule can consistently encode the wrong assumption.

Measure it against held-out cases, inspect false positives, and verify that every cited span still exists. If a detector lacks the facts required for a conclusion, it should abstain.

The replacement-card example

A trusted ordering span reports declined, while the response promises shipment. This is a strong failure label because both the external fact and conflicting claim are available.

If the tool outcome is missing, reviewers should label the case inconclusive rather than infer success or failure from the response alone.

That boundary teaches the grader when evidence supports a conclusion and when it does not.

The practical rule

Treat ground truth as a versioned, reviewable dataset with provenance. Calibrate every grader, deterministic or model-based, on examples that were not used to design it. Preserve uncertainty instead of turning it into a false score.

Perseval’s shipped safeguards and contract scenarios are not a population-level calibration study. They test bounded behavior and failure handling; they do not establish precision, recall, prevalence, or hosted-judge quality on production traffic.