Arena Elo and Bradley-Terry

Open in Colab ▶️ Run this notebook in Colab

Arena Elo and Bradley-Terry#

Track 06 - Evaluation · Notebook 05 · Runtime: ≈30 s on CPU

Prerequisites: none beyond basic probability.

Paper: Chiang et al. 2024, Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (2403.04132).

What#

Given a table of pairwise human preferences between models, fit two rating systems and compare them:

  1. Online Elo - updates each rating incrementally by the prediction error on each observed match. Fast, path-dependent.

  2. Bradley-Terry - a maximum-likelihood fit over the entire comparison matrix at once. Path-independent, the method Chatbot Arena actually uses for published leaderboards.

The two should produce approximately the same ranking on enough data. Online Elo is easier to update live; Bradley-Terry gives a closed-form confidence interval.

from llm_systems_cookbook.nb import bootstrap

import numpy as np

s = bootstrap("06_eval_05_arena_elo_bradley_terry")

Simulated preference data#

Six models with true Elo-style strengths. Generate M = 4000 random pairings; outcome probability for each match comes from the logistic P(A beats B) = 1 / (1 + 10^{(strength_B - strength_A) / 400}).

Because we simulated from known strengths, both Elo and BT should recover rankings close to the truth.

MODELS = ["alpha", "beta", "gamma", "delta", "epsilon", "zeta"]
NUM_MODELS = len(MODELS)
TRUE_STRENGTHS = np.array([1650, 1550, 1500, 1450, 1400, 1300], dtype=float)

rng = np.random.default_rng(0)
M = 4000


def simulate_match(a: int, b: int) -> int:
    '''1 if A wins, 0 if B wins.'''
    p_a = 1.0 / (1.0 + 10 ** ((TRUE_STRENGTHS[b] - TRUE_STRENGTHS[a]) / 400.0))
    return int(rng.random() < p_a)


matches: list[tuple[int, int, int]] = []
for _ in range(M):
    i, j = rng.choice(NUM_MODELS, size=2, replace=False)
    outcome = simulate_match(int(i), int(j))
    matches.append((int(i), int(j), outcome))

true_order = list(np.argsort(-TRUE_STRENGTHS))
print("true ranking (strongest first):", [MODELS[i] for i in true_order])
print(f"simulated {M} matches")

Online Elo#

Standard Elo update with K-factor 32 starting from 1500 for everyone.

def online_elo(matches: list[tuple[int, int, int]], num_models: int, K: float = 32.0,
                init: float = 1500.0) -> np.ndarray:
    r = np.full(num_models, init, dtype=float)
    for a, b, outcome in matches:
        exp_a = 1.0 / (1.0 + 10 ** ((r[b] - r[a]) / 400.0))
        score_a = float(outcome)
        r[a] += K * (score_a - exp_a)
        r[b] += K * ((1.0 - score_a) - (1.0 - exp_a))
    return r


elo_rates = online_elo(matches, NUM_MODELS)
elo_order = list(np.argsort(-elo_rates))
for i in elo_order:
    print(f"  {MODELS[i]:<8} Elo = {elo_rates[i]:7.1f}   (true = {TRUE_STRENGTHS[i]:.0f})")

Bradley-Terry#

The BT likelihood of each match (a beats b) is sigma(theta_a - theta_b). Maximise log-likelihood with gradient descent - 200 iterations of batch gradient is plenty for six models.

def fit_bradley_terry(matches: list[tuple[int, int, int]], num_models: int,
                      n_iter: int = 200, lr: float = 0.1) -> np.ndarray:
    theta = np.zeros(num_models, dtype=float)
    for _ in range(n_iter):
        grad = np.zeros_like(theta)
        for a, b, outcome in matches:
            p_a = 1.0 / (1.0 + np.exp(-(theta[a] - theta[b])))
            # d/d theta_a log P = (outcome - p_a); opposite sign for b.
            grad[a] += outcome - p_a
            grad[b] -= outcome - p_a
        theta += lr * grad / len(matches)
    # Convert theta to Elo-equivalent (multiplicative factor 400 / ln 10).
    bt_elo = theta * (400.0 / np.log(10.0))
    # Centre so the mean rating is 1500 (for visual parity with Elo).
    bt_elo += 1500.0 - bt_elo.mean()
    return bt_elo


bt_rates = fit_bradley_terry(matches, NUM_MODELS)
bt_order = list(np.argsort(-bt_rates))
for i in bt_order:
    print(f"  {MODELS[i]:<8} BT  = {bt_rates[i]:7.1f}   (true = {TRUE_STRENGTHS[i]:.0f})")

Checks#

  • Both methods recover the top-ranked model.

  • Both methods rank the six models in order that matches the true ordering on at least 4 of the 6 positions (Kendall tau-ish).

  • Elo and BT agree on the top-3 set.

def matches_fraction(pred_order: list[int], truth_order: list[int]) -> float:
    return float(sum(p == t for p, t in zip(pred_order, truth_order, strict=True)) / len(truth_order))


elo_match = matches_fraction(elo_order, true_order)
bt_match  = matches_fraction(bt_order,  true_order)

s.check(
    "elo_recovers_top_model",
    lambda: elo_order[0] == true_order[0],
    msg=f"elo top = {MODELS[elo_order[0]]}  true top = {MODELS[true_order[0]]}",
)
s.check(
    "bt_recovers_top_model",
    lambda: bt_order[0] == true_order[0],
    msg=f"bt top = {MODELS[bt_order[0]]}  true top = {MODELS[true_order[0]]}",
)
s.check(
    "elo_matches_truth_on_majority_of_positions",
    lambda: elo_match >= 4 / NUM_MODELS,
    msg=f"elo matches = {elo_match * NUM_MODELS:.0f}/{NUM_MODELS}",
)
s.check(
    "bt_matches_truth_on_majority_of_positions",
    lambda: bt_match >= 4 / NUM_MODELS,
    msg=f"bt matches = {bt_match * NUM_MODELS:.0f}/{NUM_MODELS}",
)
s.check(
    "elo_and_bt_top_3_agree",
    lambda: set(elo_order[:3]) == set(bt_order[:3]),
    msg=f"elo top-3 = {[MODELS[i] for i in elo_order[:3]]}  "
        f"bt top-3 = {[MODELS[i] for i in bt_order[:3]]}",
)

Elo trajectories#

Online Elo’s defining property is that the rating evolves one match at a time. Tracking each model’s rating over match count makes convergence visible: strong models ride up, weak ones ride down, and after enough matches the ordering stabilises near the Bradley-Terry batch solution drawn as dashed reference lines.

import matplotlib.pyplot as plt

# Replay the same matches, snapshotting each model's rating after every step.
r = np.full(NUM_MODELS, 1500.0)
traj = np.empty((len(matches) + 1, NUM_MODELS))
traj[0] = r
for t, (a, b, outcome) in enumerate(matches, start=1):
    exp_a = 1.0 / (1.0 + 10 ** ((r[b] - r[a]) / 400.0))
    r[a] += 32.0 * (outcome - exp_a)
    r[b] += 32.0 * ((1.0 - outcome) - (1.0 - exp_a))
    traj[t] = r

fig, ax = plt.subplots(figsize=(7.5, 4.4))
colors = plt.cm.tab10(np.linspace(0, 1, NUM_MODELS))
x = np.arange(len(matches) + 1)
for i in range(NUM_MODELS):
    ax.plot(x, traj[:, i], color=colors[i], linewidth=1.4,
            label=f"{MODELS[i]}  (BT={bt_rates[i]:.0f}, true={TRUE_STRENGTHS[i]:.0f})")
    ax.axhline(bt_rates[i], color=colors[i], linestyle="--", alpha=0.35)
ax.set_xlabel("match count")
ax.set_ylabel("online Elo rating")
ax.set_title("online Elo trajectories  (dashed = Bradley-Terry batch fit)")
ax.grid(alpha=0.3)
ax.legend(loc="center right", fontsize=8, framealpha=0.9)
fig.tight_layout()
plt.show()

Exercises#

  1. Bootstrap confidence intervals. Resample the match list with replacement 500 times, refit BT each time. The 2.5/97.5 percentiles of each model’s rating give a 95% CI.

  2. Ties. The Arena data includes ties. Extend the BT likelihood with a draw probability (Rao-Kupper model).

  3. Live ranking. Process the matches in time order and track how long it takes Elo to converge within 20 points of BT. This tells you when “enough votes” have arrived.

References#

  • Chiang et al. 2024 §3 for the BT formulation Chatbot Arena uses.

  • Herbrich et al. 2006, TrueSkill, for the Bayesian extension.

s.summary()
s.save()