03 - Matchmaking System Design: ELO, Glicko-2 & Queue Management

Matchmaking is a compromise between fairness and waiting

A player who waits forty seconds for a balanced match is happy; one who waits four for a match already decided in the first minute does not come back. Matchmaking lives entirely in this tension: every criterion you tighten (similar level, low ping, balanced teams) raises the queue time, every criterion you loosen lowers it and lowers the quality. There is no absolutely right value, there is the one your players tolerate, and to find it you need to be able to measure.

This article builds the three pieces needed to measure: a rating that estimates the level (Elo and Glicko-2), a queue that widens the search over time, and a criterion for forming balanced teams. Each piece has a listing that was run: the numbers in the outputs are the model's real ones, which uses fixed seeds and a synthetic population of players. After the matchmaker has formed a group, it is the server orchestrator that gives it a match.

Elo: the simplest level estimate that works

Elo assigns a number to every player. The difference between two numbers becomes a probability of winning: E(A) = 1 / (1 + 10^((RB − RA) / 400)). With 1500 against 1200, the 300-point difference gives A an 85% probability. After the match, the rating moves by K × (result − expected): whoever wins as expected gains little, whoever wins by surprise gains a lot. K is the "development coefficient" and decides how nervous the rating is.

The listing does two things. First it checks the formula on the two extreme cases of 1500 against 1200: the expected win moves the ratings by five points, the surprise one by twenty-seven. Then, to understand what K does, it simulates 6,000 matches among 40 players with a known true level (from 1000 to 1975) and measures how far the final rating strays from that level. The results of the matches are drawn from the same Elo formula: it is a model of the dynamics of K, not proof that Elo describes real players.

import random

def expected(ra: float, rb: float) -> float:
    """Expected probability that A beats B."""
    return 1 / (1 + 10 ** ((rb - ra) / 400))

def update(ra: float, rb: float, score_a: float, k: float = 32):
    ea = expected(ra, rb)
    return round(ra + k * (score_a - ea)), round(rb + k * ((1 - score_a) - (1 - ea)))

print(f"E(1500 vs 1200) = {expected(1500, 1200):.3f}")
print("A (1500) beats B (1200):", update(1500, 1200, 1))
print("B (1200) beats A (1500):", update(1500, 1200, 0))

# Effect of K: 40 players with a known true level, 6000 matches between random pairs.
# The result of each match is drawn from the same formula (model, not real data).
def run(k: int, seed: int = 1) -> float:
    rng = random.Random(seed)
    true = [1000 + 25 * i for i in range(40)]   # from 1000 to 1975
    rating = [1500.0] * 40
    for _ in range(6000):
        a, b = rng.sample(range(40), 2)
        score = 1.0 if rng.random() < expected(true[a], true[b]) else 0.0
        rating[a], rating[b] = update(rating[a], rating[b], score, k)
    # mean error on the ranking, centering the two sets (Elo is relative)
    mt, mr = sum(true) / 40, sum(rating) / 40
    return sum(abs((r - mr) - (t - mt)) for r, t in zip(rating, true)) / 40

for k in (8, 16, 32, 64):
    print(f"K={k:>2}: mean error on the true level after 6000 matches = {run(k):.0f} points")

# Output:
# E(1500 vs 1200) = 0.849
# A (1500) beats B (1200): (1505, 1195)
# B (1200) beats A (1500): (1473, 1227)
# K= 8: mean error on the true level after 6000 matches = 75 points
# K=16: mean error on the true level after 6000 matches = 38 points
# K=32: mean error on the true level after 6000 matches = 40 points
# K=64: mean error on the true level after 6000 matches = 60 points

The output has the shape of a U: with K=8 the rating chases too slowly (error 75 points after 6,000 matches), with K=64 it oscillates at every result (60), and the minimum lies between 16 and 32. This explains why federations do not use a single K. The FIDE regulations (chapter 8.3.3 of section B02) provide K=40 for those new to the list up to 30 games, K=20 as long as the rating stays under 2400 and K=10 above: a new rating must move fast, an established one must be stable. Elo, however, only knows the score: a player with 1500 after five games and one with 1500 after a thousand have the same number, and the system treats them the same.

Glicko-2: the rating with its uncertainty

Mark Glickman's Glicko-2 answers that limit by adding two numbers to each player. The rating deviation (RD) says how uncertain the estimate is: a new user starts at 1500 with RD 350, and the RD falls with games and rises again with inactivity. The volatility σ (it starts at 0.06) measures how much the player deviates from their own level: someone who alternates good and awful evenings has a high σ. The system parameter τ, between 0.3 and 1.2 according to the paper, restrains changes in volatility.

The listing implements the paper's algorithm step by step, and validates it on the example that Glickman himself computes (player 1500/RD 200 against three opponents). Differences from the implementation often seen online: the search for the starting point B for the Illinois method loops on the value of k until the function changes sign, instead of stopping at k=1, which works on the example but not in general. The second comparison is the reason RD exists: the same win is worth 175 points to a new player and 7 to a veteran. The last calculation shows inactivity.

import math

SCALE = 173.7178   # conversion factor between the Glicko scale and the internal Glicko-2 one

def g(phi: float) -> float:
    return 1 / math.sqrt(1 + 3 * phi * phi / math.pi ** 2)

def e(mu: float, mu_j: float, phi_j: float) -> float:
    return 1 / (1 + math.exp(-g(phi_j) * (mu - mu_j)))

def update(rating, rd, sigma, games, tau=0.5):
    """games: list of (opponent_rating, opponent_rd, score 1/0.5/0)."""
    mu, phi = (rating - 1500) / SCALE, rd / SCALE
    opp = [((r - 1500) / SCALE, d / SCALE, s) for r, d, s in games]
    v = 1 / sum(g(pj) ** 2 * e(mu, mj, pj) * (1 - e(mu, mj, pj)) for mj, pj, _ in opp)
    delta = v * sum(g(pj) * (s - e(mu, mj, pj)) for mj, pj, s in opp)

    a = math.log(sigma ** 2)
    def f(x):
        ex = math.exp(x)
        return (ex * (delta ** 2 - phi ** 2 - v - ex)) / (2 * (phi ** 2 + v + ex) ** 2) - (x - a) / tau ** 2
    A = a
    if delta ** 2 > phi ** 2 + v:
        B = math.log(delta ** 2 - phi ** 2 - v)
    else:
        k = 1
        while f(a - k * tau) < 0:     # look for the first k where f changes sign
            k += 1
        B = a - k * tau
    fA, fB = f(A), f(B)
    while abs(B - A) > 1e-6:           # Illinois method
        C = A + (A - B) * fA / (fB - fA)
        fC = f(C)
        if fC * fB <= 0:
            A, fA = B, fB
        else:
            fA /= 2
        B, fB = C, fC
    new_sigma = math.exp(A / 2)

    phi_star = math.sqrt(phi ** 2 + new_sigma ** 2)
    new_phi = 1 / math.sqrt(1 / phi_star ** 2 + 1 / v)
    new_mu = mu + new_phi ** 2 * (delta / v)
    return new_mu * SCALE + 1500, new_phi * SCALE, new_sigma

# Example from Glickman's paper (2012): expected 1464.06, 151.52, 0.05999
r, rd, s = update(1500, 200, 0.06, [(1400, 30, 1), (1550, 100, 0), (1700, 300, 0)])
print(f"Glickman's paper  : rating {r:.2f}, RD {rd:.2f}, sigma {s:.6f}")

# Same win against a 1500/RD 50 opponent, for a new player and for a veteran
for name, rd0 in (("new (RD 350)", 350), ("veteran (RD 50)", 50)):
    r, rd, s = update(1500, rd0, 0.06, [(1500, 50, 1)])
    print(f"{name:<18}: after one win rating {r:.0f} (+{r - 1500:.0f}), RD {rd:.0f}")

# Inactivity: with no games RD grows by sqrt(phi^2 + sigma^2) per period, capped at 350
phi = 50 / SCALE
for period in range(1, 101):
    phi = min(math.sqrt(phi ** 2 + 0.06 ** 2), 350 / SCALE)
    if period in (1, 10, 50, 100):
        print(f"after {period:>3} periods without playing: RD {phi * SCALE:.1f}")

# Output:
# Glickman's paper  : rating 1464.05, RD 151.52, sigma 0.059996
# new (RD 350)      : after one win rating 1675 (+175), RD 248
# veteran (RD 50)   : after one win rating 1507 (+7), RD 51
# after   1 periods without playing: RD 51.1
# after  10 periods without playing: RD 59.9
# after  50 periods without playing: RD 89.1
# after 100 periods without playing: RD 115.6

The first result matches the paper: 1464.05 against 1464.06 (the difference comes from the intermediate roundings of the paper's table) and a volatility of 0.059996, which the paper writes 0.05999. From the last lines you can read the design choice: a high RD makes the rating change a lot after few games (the new player who wins against a 1500 jumps to 1675), and that is exactly what is needed against beginners who are actually experts. After a hundred periods of inactivity a veteran's RD has risen from 50 to 116: the system is less sure of where they stand, and will start moving quickly again. Glickman writes that the method works best with an average of at least 10-15 games per player in a "rating period"; updating after every single game is a product choice, not the paper's, and it is worth measuring on your own data. Lichess, for example, declares that it uses Glicko-2.

The queue: a window that widens with waiting

A rating alone does not form matches. You need a queue that, given a waiting player, looks for someone of a similar level. The delicate point is "similar": a narrow window gives fair matches but leaves players at the edges of the distribution (the strongest and the weakest) stuck in the queue. The standard solution is a window that grows with the elapsed time. An ordering by rating makes searching a window fast: the sorted in-memory list in the listing does the same job as a Redis sorted set, and the equivalent command (Redis 6.2 or later) is below.

# Schema to adapt: requires Redis 6.2 or later (ZRANGE ... BYSCORE)
ZADD mm:ranked 1534 player:42                      # score = rating, member = id
HSET mm:ranked:meta:42 joinedAt 1699800000 region eu
ZRANGE mm:ranked 1434 1634 BYSCORE LIMIT 0 20     # candidates within +-100 points
ZREM mm:ranked player:42                           # returns 1 only to whoever really removes it

The rating lives in the score, the player is the member. The note on the last line matters: two workers searching for matches in parallel may see the same candidate, but ZREM returns 1 to only one, and whoever receives 0 must give up and retry. That is why the player's detail goes into a separate structure and is not serialized inside the member: replacing a JSON member means scanning all of them to find it again. The command ZRANGE has complexity O(log N + M), where M is the number of elements returned (Redis documentation).

The following listing simulates 10 minutes of 1-vs-1 queue, with one player every two seconds on average and levels drawn from a normal distribution (mean 1500, deviation 300). I try four policies: a fixed window of ±25, two that widen by 5 and 25 points per second, and a fixed wide one of ±200. At the end, the same file solves a second problem: splitting ten players into two teams of five.

import bisect
import itertools
import random

class Queue:
    """In-memory equivalent of a sorted set: list sorted by (rating, id)."""
    def __init__(self):
        self._keys: list[tuple[float, int]] = []
        self.joined: dict[int, int] = {}

    def add(self, pid: int, rating: float, now: int):
        bisect.insort(self._keys, (rating, pid)); self.joined[pid] = now

    def remove(self, pid: int, rating: float):
        self._keys.remove((rating, pid)); del self.joined[pid]

    def in_range(self, lo: float, hi: float):          # like ZRANGE ... BYSCORE
        a = bisect.bisect_left(self._keys, (lo, -1))
        b = bisect.bisect_right(self._keys, (hi, 10 ** 9))
        return self._keys[a:b]

def simulate(base: float, growth: float, seed: int = 3, seconds: int = 600):
    rng = random.Random(seed)
    q, pid = Queue(), 0
    waits, gaps = [], []
    ratings: dict[int, float] = {}
    for now in range(seconds):
        if rng.random() < 0.5:                             # one player every 2 s on average
            pid += 1; ratings[pid] = rng.gauss(1500, 300); q.add(pid, ratings[pid], now)
        for p in sorted(q.joined, key=lambda x: q.joined[x]):   # oldest first
            if p not in q.joined:
                continue
            r = ratings[p]
            window = base + growth * (now - q.joined[p])        # window that widens with the wait
            others = [(rr, i) for rr, i in q.in_range(r - window, r + window) if i != p]
            if others:
                rr, i = min(others, key=lambda x: abs(x[0] - r))
                waits += [now - q.joined[p], now - q.joined[i]]; gaps.append(abs(rr - r))
                q.remove(p, r); q.remove(i, rr)
    return len(gaps), sum(waits) / len(waits), sum(gaps) / len(gaps), len(q.joined)

print("initial window + growth per second -> matches, mean wait, mean rating gap, still in queue")
for base, growth in ((25, 0), (25, 5), (25, 25), (200, 0)):
    n, w, gp, left = simulate(base, growth)
    print(f"+-{base:>3} + {growth:>2}/s: {n:>3} matches, wait {w:5.1f} s, gap {gp:5.1f}, in queue {left:>2}")

# Balancing 10 players into 5v5: greedy against an exhaustive search of all 126 splits
rng = random.Random(11)
def greedy(players):
    t1, t2 = [], []
    for p in sorted(players, reverse=True):
        if len(t1) < 5 and (sum(t1) <= sum(t2) or len(t2) == 5): t1.append(p)
        else: t2.append(p)
    return abs(sum(t1) - sum(t2))
def best(players):
    return min(abs(2 * sum(c) - sum(players)) for c in itertools.combinations(players, 5))
gg = bb = 0
for _ in range(200):
    pl = [round(rng.gauss(1500, 300)) for _ in range(10)]
    gg += greedy(pl); bb += best(pl)
print(f"mean gap between team sums over 200 lobbies: greedy {gg / 200:.1f}, exhaustive {bb / 200:.1f}")

# Output:
# initial window + growth per second -> matches, mean wait, mean rating gap, still in queue
# +- 25 +  0/s: 145 matches, wait  24.0 s, gap  13.5, in queue 20
# +- 25 +  5/s: 153 matches, wait   7.5 s, gap  61.5, in queue  4
# +- 25 + 25/s: 154 matches, wait   3.7 s, gap 123.0, in queue  2
# +-200 +  0/s: 154 matches, wait   4.1 s, gap  95.7, in queue  2
# mean gap between team sums over 200 lobbies: greedy 138.2, exhaustive 12.2

The first part of the output is the compromise of the opening paragraph, in numbers. The narrow fixed window gives the minimum rating gap (13.5 points) but 24 seconds of mean wait and twenty people still in the queue. The wide fixed window almost removes the wait (4.1 s) at the cost of a gap seven times larger (95.7). The window that grows by 5 points per second sits in the middle: 7.5 s and 61.5. Everything depends on what your game wants, and that is why the policy is a parameter to tune on real data, not a constant in the code. I chose round numbers for the model; yours will depend on the density of players in the queue.

The second part is a lesson about teams. "Greedy" balancing (sort by rating and give each player to the team with the lowest sum) seems reasonable, but over 200 random lobbies it leaves a mean difference of 138 points between the sums of the two teams. Trying all 126 possible splits of ten players into 5 and 5 costs few operations, and brings the mean difference down to 12. With lobbies this small there is no reason to avoid exhaustive search.

SBMM or CBMM: which of the two constraints you give up

Skill-based matchmaking (SBMM) favors balance of level; connection-based matchmaking (CBMM) favors network proximity. The public debate makes them two factions, but at the level of code they are two weights in the same candidate scoring function.

The table is read not by column but by row: every choice buys something and pays for it elsewhere. The one that suits you depends on the density of the population. With thousands of people in the queue even a tight SBMM quickly finds an opponent; with a few dozen, a pure SBMM leaves players stuck and it is better to widen the window or separate the modes. When the population is scarce, the safest thing to give up is level and not the network: an unbalanced match gets finished, a match with an unmanageable ping does not.

Smurfs, backfill and metrics

A smurf is a strong player on a new account. Glicko-2 already handles them better than Elo: the new account has RD 350, so the first wins move it by many points (this is the +175 in the output), and its rating adjusts to the true level much faster. What is missing is a detector for the edge cases, and the thresholds of a detector (win rate, aim accuracy, learning curve) are not imported from an article: they are derived from your players' data and validated against known cases. Whoever publishes a fixed threshold hands cheaters the boundary not to cross.

Backfill is the opposite case: a match in progress has lost a player and looks for a replacement in the queue. It has different rules from the normal search, because the window must look at the average level of those already playing, and the person who joins did not see the start. If you use GameLift, the callback onUpdateGameSession is exactly the channel on which the service communicates the FlexMatch backfill to the game server. As for the quality of the system, these are the five quantities I would read first.

• 95th percentile of queue wait: the long tail matters more than the mean.
• Rating gap between the two teams, per match: the effective quality of the matches.
• Exits in the first part of the match: whoever leaves immediately is saying it was unbalanced.
• Distance from the 50% win rate per level band: the proof that the rating is calibrated.
• Mean latency to the assigned server.

The five items measure two different things: the second, the third and the fourth say how fair the matches are, the first and the last the price you pay in time and in network. I do not give you thresholds because there are no universal ones: the starting value is today's, and the useful question is whether a change of policy moves them in the direction you wanted. A comparison between two policies, with the same data for a week, is worth more than any target.

What you can do now

You can compute an Elo update and explain why K has an optimal value, not a maximum value. You can implement Glicko-2 and verify it on the paper's example, and you can say when Elo's simplicity is the better fit. You can design a queue with a variable window, measure its trade-off between wait and gap, and not leave team balancing to a greedy algorithm when a few operations are enough to find the optimal split. The next step is to make the game feel smooth despite the network: real-time state synchronization. For a ready-to-use matchmaker, there is Open Match.