LLM-as-judge bias#
Track 06 - Evaluation · Notebook 04 · Runtime: ≈30 s on CPU
Prerequisites:
06_eval/01(perplexity from scratch).Paper: Zheng et al. 2023, Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (2306.05685).
What#
Using one LLM to score another’s output is cheap and fast but has three well-known biases:
Position bias. When presented with two answers in a fixed (A, B) order, judges preferentially vote for A (or B) regardless of content.
Verbosity bias. Longer answers get higher scores, independently of whether the extra length adds information.
Self-enhancement bias. A judge scores outputs from its own model family higher than those from other families.
We simulate a pool of pairwise comparisons, inject each bias, and measure it. The mitigation standard from Zheng 2023 is position swapping: evaluate each pair twice, once as (A, B) and once as (B, A), then only count the pair as decisive if both orderings agree.
from llm_systems_cookbook.nb import bootstrap
import numpy as np
s = bootstrap("06_eval_04_llm_as_judge_bias")
The setup#
200 pairs of candidate answers. Half are A-stronger, half are B-stronger (ground truth). Each candidate has a length drawn from a distribution, and we simulate a biased judge whose preference is a logistic function of: the true quality gap, the length gap, and a fixed position bonus for whichever side is shown first.
N = 200
rng = np.random.default_rng(0)
true_gap = rng.normal(0, 1.0, size=N) # positive: A is better
# Ground truth: A wins iff true_gap > 0.
true_winner = (true_gap > 0).astype(int) # 0 = A wins, 1 = B wins? flipped below
true_winner = np.where(true_gap > 0, 0, 1) # 0 means A is the correct winner
len_a = rng.integers(50, 800, size=N)
len_b = rng.integers(50, 800, size=N)
def sigmoid(x: np.ndarray) -> np.ndarray:
return 1.0 / (1.0 + np.exp(-x))
def biased_judge(true_gap: np.ndarray, len_a: np.ndarray, len_b: np.ndarray,
*, position_bonus_for_first: float, verbosity_weight: float,
quality_weight: float = 1.0) -> np.ndarray:
'''Return judge's preferred answer index in the pair (0=A, 1=B).'''
# Score from judge's perspective: higher means A is preferred.
raw = quality_weight * true_gap + verbosity_weight * (len_a - len_b) + position_bonus_for_first
prob_a = sigmoid(raw)
# Threshold at 0.5 deterministically.
return (prob_a < 0.5).astype(int)
Position bias measurement#
Present each pair as (A, B) and again as (B, A). A judge with no position bias should pick the same winner both times.
# Judge with strong position-for-first bias.
verdicts_AB = biased_judge(
true_gap=true_gap, len_a=len_a, len_b=len_b,
position_bonus_for_first=1.0, verbosity_weight=0.0,
)
# Swap the pair: first argument becomes B, second becomes A.
verdicts_BA = 1 - biased_judge(
true_gap=-true_gap, len_a=len_b, len_b=len_a,
position_bonus_for_first=1.0, verbosity_weight=0.0,
)
# "Flip rate": fraction of pairs where the judge changes its verdict
# purely because of position.
flip_rate = (verdicts_AB != verdicts_BA).mean()
print(f"position-swap flip rate = {flip_rate:.1%} (unbiased judge would give 0%)")
# Decisive rate after swap-mitigation: pair counts only if both orderings agree.
decisive_mask = verdicts_AB == verdicts_BA
decisive_rate = decisive_mask.mean()
accuracy_on_decisive = (verdicts_AB[decisive_mask] == true_winner[decisive_mask]).mean()
print(f"decisive rate with swap = {decisive_rate:.1%}")
print(f"accuracy on decisive = {accuracy_on_decisive:.3f}")
# Bare accuracy without swap-mitigation.
bare_accuracy = (verdicts_AB == true_winner).mean()
print(f"bare accuracy (AB only) = {bare_accuracy:.3f}")
Verbosity bias measurement#
Same framework, different bias: the judge prefers whichever answer is longer, independently of quality. Mitigation: regress out length from the verdicts.
verdicts_verbose = biased_judge(
true_gap=true_gap, len_a=len_a, len_b=len_b,
position_bonus_for_first=0.0, verbosity_weight=0.003, # strong length preference
)
# How often does the judge side with the longer answer regardless of truth?
longer_side = (len_a < len_b).astype(int) # 1 if B longer
align_with_longer = (verdicts_verbose == longer_side).mean()
print(f"verbose-biased judge agrees with longer side: {align_with_longer:.1%}")
print(f" (unbiased judge would agree ~50%)")
# Compare against a non-verbose judge (same gap, zero verbosity weight).
verdicts_unbiased = biased_judge(
true_gap=true_gap, len_a=len_a, len_b=len_b,
position_bonus_for_first=0.0, verbosity_weight=0.0,
)
unbiased_accuracy = (verdicts_unbiased == true_winner).mean()
verbose_accuracy = (verdicts_verbose == true_winner).mean()
print(f"accuracy: unbiased={unbiased_accuracy:.3f} verbose-biased={verbose_accuracy:.3f}")
s.check(
"position_biased_judge_flips_substantially",
lambda: flip_rate > 0.15,
msg=f"flip rate = {flip_rate:.1%}",
)
s.check(
"swap_mitigation_raises_accuracy",
lambda: accuracy_on_decisive > bare_accuracy,
msg=f"bare={bare_accuracy:.3f} swap-decisive={accuracy_on_decisive:.3f}",
)
s.check(
"verbose_judge_biased_toward_longer_side",
lambda: align_with_longer > 0.60,
msg=f"agrees with longer = {align_with_longer:.1%}",
)
s.check(
"verbosity_bias_degrades_accuracy",
lambda: verbose_accuracy < unbiased_accuracy,
msg=f"unbiased={unbiased_accuracy:.3f} verbose={verbose_accuracy:.3f}",
)
s.check(
"swap_decisive_rate_meaningfully_less_than_one",
lambda: 0.1 < decisive_rate < 0.95,
msg=f"decisive rate = {decisive_rate:.1%} (biased judge should flip enough "
f"pairs to make decisive rate < 1, but not so much that nothing is decisive)",
)
Bias summary#
One chart summarises both biases: the position-swap flip rate for the position-biased judge, and the longer-side agreement rate for the verbosity-biased judge, side-by-side against their unbiased baselines. A reference line at 50% marks “no preference” for the longer-side agreement; any real lift above the baseline is the bias signal.
import matplotlib.pyplot as plt
# Unbiased-judge baselines, computed by running the same harness with all bias knobs zeroed.
v_AB_unb = biased_judge(true_gap=true_gap, len_a=len_a, len_b=len_b,
position_bonus_for_first=0.0, verbosity_weight=0.0)
v_BA_unb = 1 - biased_judge(true_gap=-true_gap, len_a=len_b, len_b=len_a,
position_bonus_for_first=0.0, verbosity_weight=0.0)
flip_unb = (v_AB_unb != v_BA_unb).mean()
align_unb = (verdicts_unbiased == longer_side).mean()
labels = ["position-swap flip rate", "agrees-with-longer rate"]
unbiased = [flip_unb, align_unb]
biased = [flip_rate, align_with_longer]
x = np.arange(len(labels)); w = 0.38
fig, ax = plt.subplots(figsize=(6.8, 4.0))
ax.bar(x - w/2, unbiased, w, color="tab:gray", label="unbiased judge")
ax.bar(x + w/2, biased, w, color="tab:red", label="biased judge")
ax.axhline(0.5, color="black", linestyle=":", alpha=0.5, label="no-preference (50%)")
for xi, (u, b) in enumerate(zip(unbiased, biased)):
ax.text(xi - w/2, u, f"{u:.1%}", ha="center", va="bottom", fontsize=9)
ax.text(xi + w/2, b, f"{b:.1%}", ha="center", va="bottom", fontsize=9)
ax.set_xticks(x); ax.set_xticklabels(labels)
ax.set_ylim(0, 1.0); ax.set_ylabel("rate")
ax.set_title("judge bias: position and verbosity effects")
ax.legend(loc="upper left", fontsize=9)
fig.tight_layout(); plt.show()
Exercises#
Self-enhancement bias. Add a third bias: the judge prefers answers whose style matches its own. Simulate it by giving the model a “style fingerprint” column in
len_a, len_b-style data and seeing how often it sides with same-fingerprint candidates.Length-debiased scoring. Fit a logistic regression of the judge’s verdict on
len_a - len_bandtrue_gap, then use the residual (verdict minus length-predicted-verdict) as the corrected signal.Inter-judge agreement. Add a second judge with different bias coefficients. Cohen’s kappa between the two gives you a sense of how much their biases overlap vs differ.
References#
Zheng et al. 2023, §3.3 for the position-swap mitigation.
Panickssery et al. 2024, LLM Evaluators Recognize and Favor Their Own Generations (arxiv 2404.13076) for self-enhancement bias.
s.summary()
s.save()