The first RTP alert was replaced in six days: from ±1.5 points to "can't judge yet"
I’ve written two versions of RTP monitoring for one project. In the first, the report defaulted to seven days, flagging a game and currency after 10,000 bets if actual RTP was more than 1.5 percentage points off the live version’s registered value. Rerun on the project’s demo slot with equal stakes and nothing wrong, the rule fires 57.9% of the time in a window that has just reached 10,000 bets.
Six days later the code switched to sealed daily windows and confidence intervals using the game’s variance. A model without a qualifying registration returns model_unavailable and keeps actual RTP without judging a breach. The demo slot itself failed Monte Carlo qualification with seeds fixed in advance. The project hasn’t been deployed to production yet; both versions have only run in development.
What the first version got wrong
The demo slot has three reels, each with five symbols weighted 1, 3, 6, 12 and 28 out of 50, and only three of a kind pays: 500, 80, 25, 10 or 4.3 times the stake. Theoretical RTP is 95.78688%, and the standard deviation of the payout multiple per round is 2.859.
At 10,000 bets, the standard error of observed RTP is 2.859/100, about 2.86 points. A 1.5-point band is just over half a standard error, so ordinary variance crosses it. More bets mean fewer false alarms: under the normal approximation the rate is about 9.7% once seven days add up to 100,000 bets, and a 1% rate in a single window needs about 241,000 rounds.
A threshold that looks strict rings six times in ten just after a window reaches 10,000 bets, and when it rings it can’t tell luck from a fault. The groundwork is in “Per-spin SD for a slot with a 30-step route bonus” and “Setting tolerances for Monte Carlo tests”.
How the replacement decides
Rounds record the game server’s maths version and SHA-256 hash; the backend assigns paid or free mode. These fields are immutable, though the legacy interface can omit version and hash. Testing groups by game, maths version, hash, currency and mode over a UTC day. Each group registers when its first round opens; new groups join throughout the day.
Kish’s unequal-weight formula defines effective sample size as total stake squared over the sum of squared stakes, with a model-specific minimum of at least 10,000. The largest squared stake must contribute at most 1% of the squared-stake sum, limiting its variance contribution. Paid rounds use the wager; free rounds use the grant’s nominal stake. Standard error is the square root of model variance times the squared-stake sum, divided by total stake.
Testing several hypotheses increases the chance of false alerts, so Bonferroni correction sets the daily family’s nominal rate at 1%. The correction can be conservative; each test still needs a valid normal approximation. A breach means the theoretical mean falls outside the interval. Each result is one of ready, insufficient_sample, incomplete_window or model_unavailable, and once sealed it can’t be rewritten, not even by the application’s own database role.
The replacement removed the old rule from the code.
The demo model fails qualification
These intervals rely on the normal approximation. The demo model’s Monte Carlo false-alert check uses seeds fixed in advance: 10,000 trials, each with 10 hypotheses at 10,000 equal-stake rounds, counting any breach as a family false alert. Its one-sided 95% Wilson upper bound must be at most 1.5%, against a nominal rate of 1%.
The demo slot’s registration records 406 false alerts and a 4.4% upper bound, and fails the criterion. Its top prize has a probability of one in 125,000, so most 10,000-round windows miss it; the normal approximation understates tail deviations for this distribution. The migration marks the model unverified: complete windows return model_unavailable and keep actual RTP.
Rerunning the project’s slow test on the Mac gave the same 406 in 10,000 and a 4.40% upper bound. The power check passed: positive and negative 5-point biases at 100,000 effective rounds were detected 98.96% and 98.39% of the time respectively, against a required 90%.
I reran the same distribution in Python. Different seeds shift the figures slightly, with the same conclusion. Rerunning in a python:3.13.18-slim container under OrbStack gives:
"""The project's three-reel demo slot, checked two ways.
Reel weights 1, 3, 6, 12, 28 out of 50 on each of three reels; three of a kind
pays 500, 80, 25, 10 or 4.3 times the stake, anything else pays nothing.
"""
import math
import numpy as np
weights = np.array([1, 3, 6, 12, 28]) / 50
pays = np.array([500, 80, 25, 10, 4.3])
p_win = weights ** 3
prob = np.append(p_win, 1 - p_win.sum()) # last outcome: no win
mult = np.append(pays, 0.0)
mean = float(mult @ prob)
sd = math.sqrt(float((mult - mean) ** 2 @ prob))
print(f"RTP {mean:.7f} per-round SD {sd:.4f}")
rng = np.random.default_rng(42)
TRIALS = 10_000
def rtp_samples(n, shape):
return rng.multinomial(n, prob, size=shape) @ mult / n
# 1. The first rule: 10,000 rounds, alert if |actual - theory| > 1.5 points.
obs = rtp_samples(10_000, TRIALS)
print(f"old rule, 10,000 rounds: false alerts {np.mean(np.abs(obs - mean) > 0.015):.4f}"
f" (normal approx {math.erfc(0.015 / (sd / 100) / math.sqrt(2)):.4f})"
f" below theory {np.mean(obs < mean):.4f}")
print(f"rounds for 1% false alerts on a 1.5-point band: {math.ceil((2.5758 * sd / 0.015) ** 2):,}")
# 2. The replacement: a daily family of 10 hypotheses, two-sided, Bonferroni at 1%.
z = 3.2905267 # normal quantile 1 - 0.01 / (2 * 10)
for n in (10_000, 50_000, 200_000):
fam = rtp_samples(n, (TRIALS, 10))
se = sd / math.sqrt(n)
rate = np.mean(np.any(np.abs(fam - mean) > z * se, axis=1))
print(f"family of 10, n = {n:>7,}: any false alert {rate:.4f} (target 0.0100)")
RTP 0.9578688 per-round SD 2.8591
old rule, 10,000 rounds: false alerts 0.5785 (normal approx 0.5998) below theory 0.5211
rounds for 1% false alerts on a 1.5-point band: 241,046
family of 10, n = 10,000: any false alert 0.0386 (target 0.0100)
family of 10, n = 50,000: any false alert 0.0186 (target 0.0100)
family of 10, n = 200,000: any false alert 0.0122 (target 0.0100)
This simulation gives 57.9% false alerts for the first rule, against 60.0% from the normal approximation. The replacement formula gives 3.86% family false alerts at 10,000 equal-stake rounds, close to the recorded 4.06%. At 200,000 rounds it gives 1.22%, with a Wilson upper bound of 1.41%, below 1.5%; this exploratory result does not qualify a revised model.
What the standards ask for
Appendix A.5.2 of GLI-19 version 4.0 ties monitoring frequency, measurement periods and acceptable variance to a game’s volatility and volume of play. The UK Gambling Commission’s testing strategy calls volatility a key parameter for each game’s tolerance, and its live RTP guidance says tolerances are wider with limited play and narrow as play builds up. The first version’s fixed tolerance reflected neither factor; these documents do not prescribe the method used here.
What is still open
At model_unavailable, actual RTP remains available without a confidence interval or breach alert. When the normal approximation cannot be trusted, I would rather the system said it cannot judge yet. This automated decision misses a wrong pay table or version; recorded maths versions and hashes support separate checks.
A higher minimum effective sample or another interval method could be assessed, then requalified; the current code adopts neither.
Notes
- The first report defaults to seven days, flagging deviations beyond 1.5 points after 10,000 bets per game and currency; the demo slot has per-round SD 2.859 and 57.9% false alerts in the equal-stake simulation at exactly 10,000 bets.
- Under the normal approximation the same band gives about 9.7% at 100,000 bets, and a 1% rate in a single window needs about 241,000 rounds.
- The replacement registers groups per UTC day, with a Kish effective sample floor of 10,000, at most 1% contributed by the largest squared stake, and a daily nominal Bonferroni rate of 1%.
- The demo slot’s Monte Carlo record is 406 in 10,000 with a 4.4% upper bound (rerunning the project’s test gives the same); it failed qualification and complete windows get model_unavailable.
- At model_unavailable, actual RTP remains available without breach alerts; versions and hashes support separate checks.
Further reading
- GLI, “GLI-19 Standards for Interactive Gaming Systems v4.0” (3 November 2026): a test laboratory standard; Appendix A.5.2 ties monitoring thresholds to volatility and volume of play.
- UK Gambling Commission, “Testing strategy: live RTP monitoring”: the regulator’s companion document, naming volatility as a key parameter for tolerance.
- UK Gambling Commission, “Live return to player performance monitoring”: regulator guidance on tolerances narrowing with volume.
- Wikipedia, “Bonferroni correction”: encyclopaedia entry on dividing the significance level across tests.
- Wikipedia, “Design effect”: encyclopaedia account of Kish’s 1965 work on design effect and effective sample size, including the unequal-weight formula.
- Wikipedia, “Binomial proportion confidence interval”: encyclopaedia entry including the Wilson score interval.
- Wikipedia, “Berry–Esseen theorem”: encyclopaedia entry linking normal approximation error to the third absolute moment.
Ideas and technical judgement by Sheng; drafted with Claude · Example run with Python 3.13.18 and numpy 2.5.3 (cloud Linux and an OrbStack container), project Monte Carlo test rerun with Go 1.27.3; tools: Claude Code (Opus 5.5, Fable 5.1), Codex (gpt-6-astra), Grok CLI.