An LLM judge evaluates outputs at scale, and human experts should label only where it is least sure. The judge’s natural escalation signal, the spread of its predicted label distribution, conflates two uncertainties: aleatoric, real disagreement in the expert pool, which more labels cannot reduce, and epistemic, the judge’s ignorance of what the experts would say, which even one label reduces. A small Bayesian model separates them: a regression on the labels already collected learns how far to trust a black-box judge’s prediction on each item, and both components then follow as closed-form expressions, with no sampling and no further judge calls.

On a real LLM judge scored against exactly known truth (500 three-sided dice), the two estimated components isolate: estimated aleatoric tracks the true disagreement and not the judge’s error, estimated epistemic tracks the judge’s error and not the disagreement. The judge’s own stated confidence is no guide to its actual error. On real human disagreement (ChaosNLI, 100 annotations per item), ranking by the epistemic estimate removes 83% more error than ranking by total uncertainty for the same expert labels, though simply escalating the least-labelled items does as well there.

When an LLM judge makes a wrong safety decision, does its own uncertainty know? Under the bias–variance decomposition, errors are either variance errors that are detectable by resampling, or bias errors where the judge is confidently and repeatably wrong. We find it is mostly the latter on frontier judges. Wrong answers are usually stable under resampling, the same wrong winner returns when you re-draw the scores. GPT-5.4’s errors are 70% stable across all categories and 86% stable on Safety, so a disagreement-based detector can catch at most 14% of its Safety errors. We show there is a ceiling on what single-judge disagreement can catch, and this ceiling falls as judge models get stronger, with the effect strongest on Safety.

Why this matters

LLM Judges are increasingly deployed as safety monitors, the backstop that is supposed to stop a model from producing unsafe outputs, where the tolerance for error is close to zero. When Anthropic released Fable 5, it was implemented as Mythos 5 with safety guardrails on top to prevent unsafe use, however, these are not guaranteed to prevent unsafe use in all scenarios. Therefore, a key problem in AI Safety research is how we can improve the safety monitors to control AI models. As we recommended in our ICML workshop paper, a cheap improvement to standard LLM as a Judge is to resample the same model to reduce variance errors. Another approach may be to sample independent models. Kohli found that sampling different models shares bias errors too, concluding that nine judges are worth roughly two independent votes. A detector built only on disagreement can reduce errors, but can give false reassurance precisely on the inputs an adversarial model would choose.

How we measure it

On RewardBench 2 each example has four candidate responses, with one being the correct one, and a judge scores all four. We collected eight independent score samples per candidate response, which gives an estimation of the distribution of the judges’ rating. We run this for two frontier judges, GPT-5.4 and Sonnet 4.6, on the same set of around 1,750 examples, plus three weaker judges: GPT-5.4-mini, Haiku 4.5, and GPT-5.4-nano. A judge’s predicted winner is the response with the highest mean score across its eight samples; if that is not the correct candidate, the judge has made an error.

What we measure is whether an error survives resampling. For each error we bootstrap the eight existing samples: resample the scores per response with replacement, recompute the winner, and repeat 1,000 times. We call the fraction of those draws that reproduce the same wrong winner $p_\text{stable}$. This deliberately resamples the eight scores we already have rather than making fresh judge calls, because those eight samples are exactly the information an ensemble or disagreement detector has at deployment. A result about their stability is therefore a result about what any such detector can ever see. We label a non-tie error bias-dominated when $p_\text{stable} \geq 0.9$, with sensitivity reported at 0.75 and 0.95 (ties, where no single response wins, count as errors but are analysed separately). The bias share is the fraction of a judge’s non-tie errors that are bias-dominated. A disagreement-based detector can only flag an error it sees move under resampling, so the most it can ever catch is the variance-dominated fraction. We call 1 − bias share the recall ceiling, an upper bound on the recall of any detector built from single-judge disagreement.

Results

Most errors are bias-dominated, and this peaks on safety tasks

For both frontier judges, most errors survive resampling.

The bias share is not uniform across the benchmark, it runs from 56% on Math to 86% on Safety for GPT-5.4. It is highest on Safety, the category where we would least want a blind monitor.

Stronger judges have a larger bias share

Within a model family, the bias share rises with model size.

From nano to mini to the full model the detector ceiling falls, 72% to 50% to 30%. This does not mean larger models make worse judges, as they are still less likely to make errors in the first place. We highlight these results to emphasise that the shape of errors are very different as model capability increases.

Different models share the same blind spot

We observe when a frontier judge is wrong the other is also wrong more than half the time, roughly three times its base rate. A panel of judges does not buy as many independent votes as its size suggests as the judges tend to fail on the same inputs.

Some of this can be attributed to task difficulty, but the figure shows this isn’t entirely correlated with difficulty. Within every difficulty band they are wrong together more often than independence predicts. Independence is the two judges’ error rates within the band multiplied together, the rate you would expect if one of them being wrong told you nothing about the other. We measure difficulty by how many of the three weaker judges (GPT-5.4-mini, Haiku 4.5, GPT-5.4-nano) also get it wrong.

Comparing difficulty-adjusted errors across model families shows that blind spots overlap the least across model providers.

A ceiling, not a weak signal

Our earlier paper found that a judge’s score variance predicts incorrectness with an AUC of about 0.60, a weak but positive signal. A low AUC could mean we just have a noisy estimator, however, the bias share shows that for most errors there is nothing in the disagreement to detect. Even the best resample signal we could build, the winner’s instability ($1 - p_\text{stable}$), only reaches AUC 0.78 to 0.81, and it is capped the same way; it cannot separate a stable wrong winner from a confidently correct one. Therefore, we present a limit to the class of disagreement solutions.

What this means for safety

Averaging more draws has limits, and polling different judges still has limits beyond capability. Two judges from the same family share their blind spots far more than two different providers do. So escalating from mini to full within a family is the worst case, it sends the monitor’s hardest cases to the judge whose blind spots line up most closely with its own. The trigger is weak also, escalating the most uncertain fifth of cases catches only about a third of the frontier judge’s bias errors.

An adversary who can push an input into the stable region can exploit a confidently-wrong monitor for free. It gives the most reassurance on exactly the inputs an adversary would choose. This is a risk that should be considered when building safety monitors.

The same shape of claim runs through what we think is the natural sequel. The uncertainty signal you would most want to trust is blind exactly where an adversary would operate. A judge that is confidently wrong on a steered input, and a model that sandbags when it senses it is being evaluated, are the same problem, the behaviour you care about hides where your detector sees calm.

When evaluating AI systems, we have to make a trade-off between quality and scalability. We can either:

1. Ask human experts to analyse the AI’s actions and outputs to ensure they comply with the AI controller’s definition of quality and safety. Naturally, this doesn’t scale very well.
2. Use an automated system, including classic NLP methods (e.g. n-gram overlap metrics like BLEU/ROUGE, or named entity recognition) or modern LLM-based methods, “LLM as a Judge”, to rate the quality of an AI’s actions and outputs. This scales much better than human experts; however, agreement between human experts and automated systems is well-documented as an issue.

We have been working on improving automated quality checks using LLM as a Judge. Other work has shown that fine-tuning dedicated reward models can prove very effective at performing better on reward model benchmarks (e.g. Skywork Reward Model dominated the RewardBench leaderboard). However, the problems I’m working on at Composo AI often leave us with customers asking for state-of-the-art automated evaluation quality, without requiring their data to be fine-tuned on (which they often don’t want to give to a third party, and would require a large amount of specific data they don’t have). Also, fine-tuning is quickly becoming a thing of the past in the frontier lab offerings (OpenAI deprecated their fine-tuning API), meaning we are often stuck on open-weight models, which have a performance impact too.

With this in mind, we have been exploring how we can improve the quality of automated LLM as a Judge evaluations, using only techniques that can be applied to standard model inference APIs. I had been using a set of these techniques to improve evaluation on real customer data; for the paper we consolidated them and ran standardised experiments on RewardBench 2 (paper and code above). We tested four: ensembling, a one-line task-specific criterion, a calibration example, and adaptive escalation to a bigger model when the cheap one looked unsure. The two simplest, ensembling and criteria, dominated the cost-accuracy Pareto frontier, taking accuracy from 72.3% to 85.8%. The most promising technique, ensembling, controlled the variance term of the bias-variance error decomposition of LLM Judge errors, where the improvement was greatest in the smallest models. From an AI Safety perspective, safety guardrails are bounded by how good, fast and cheap automated evaluations are, so this improvement can also enable more reliable safety guardrails.

Ensembling the same model averages away variance but not bias, because every sample shares the model’s blind spots. Decomposing the error in the ensemble’s outputs should give a better understanding of uncertainty, and studying the bias error is what I’m thinking about next.

I will be at ICML in Seoul this July. If you are around and want to talk evals, judge reliability, or any of the above, find me on LinkedIn or X.