GPT-3 really did use a bloom filter to catch contaminated eval data, Llama 2's own paper found 16% of MMLU had leaked into its training set, and running the actual bootstrap-CI gate on a real 200-example eval shows a genuinely improved model getting blocked as a false regression purely from sampling noise

This series already has a full field guide to what each eval metric measures and how it lies: judge bias, calibration, RAG’s two-component trap, all of it. This post is the layer underneath that guide, the pipeline infrastructure that decides whether any of those metrics are even trustworthy enough to act on: was the eval set contaminated by the training data in the first place, was it large enough to actually detect the difference you’re looking for, and does the automated promotion gate that reads the resulting number actually make the right call. All three questions have real, precise, computable answers, and running the actual gate at a realistic small sample size produces a result worth seeing directly: a genuinely, truly improved model gets blocked as a regression, not because the gate is broken, but because it’s working exactly as designed on too little data.

Contamination: a real bloom filter, not a metaphor#

GPT-3’s own paper states directly that it used 13-gram overlap detection with a bloom filter to measure and bound contamination between its training corpus and the benchmarks it would later be evaluated on: for each benchmark example, extract every 13-token sequence, and check whether that sequence appears anywhere in the training data. A bloom filter makes this computationally tractable at pretraining-corpus scale, trillions of tokens, by trading a small, known false-positive rate for the ability to check billions of sequences in bounded memory, rather than storing every training n-gram explicitly. GPT-4’s technical report raised the matching bar to 40-grams, a stricter, less falsely-triggered standard, though later work using Apache Spark for exact-match collision counting suggests the field has moved past bloom filters’ probabilistic bound toward exact overlap computation where compute allows it.

A small, direct version of the same check, minimal enough to actually run:

def ngrams(tokens, n=13):
    return set(tuple(tokens[i:i+n]) for i in range(len(tokens) - n + 1))

train_ngrams = ngrams(training_data_tokens, n=8)
eval_ngrams = ngrams(eval_example_tokens, n=8)

overlap = eval_ngrams & train_ngrams
if overlap:
    flag_as_contaminated(eval_example, overlap_count=len(overlap))

Run against a genuinely clean eval example, zero shared n-grams, not flagged. Run against an example that shares a long run of text with the training corpus, real, non-zero overlap, correctly flagged. The mechanism is exactly this simple. What isn’t simple is how much this actually matters at real scale: Meta’s own Llama 2 report found that 16% of MMLU’s items overlapped with its training data, and some of those overlaps exceeded 80% of the tokens in the example matching exactly. That’s not a hypothetical risk stated to justify a section in a paper, it’s a real, disclosed, measured contamination rate in a benchmark used to report the headline capability numbers of a real, shipped model.

The real, documented limitation worth knowing: n-gram matching only catches exact or near-exact text overlap, not paraphrase. A training document that rewords a benchmark question with different vocabulary and sentence structure, same underlying content, passes an n-gram filter cleanly while still teaching the model the answer. This gap is exactly what’s driven contamination-detection research past simple overlap counting: Min-K% Prob checks whether a model assigns suspiciously high probability to an example’s least-likely tokens, a black-box signal that doesn’t require access to the training corpus at all, and DyePack (2025) goes further, deliberately inserting backdoor-style canary patterns into benchmark data so contamination can be provably flagged after the fact, with a stated false-positive guarantee, rather than merely inferred from a suspicious probability score.

The real power calculation behind “your eval is too small”#

The field guide’s own version of this question uses a simplified standard-error heuristic. Here’s the full calculation properly, the one that actually produces a specific required sample size rather than a rule of thumb, using the standard two-proportion power formula:

$n = \frac{2(z_{\alpha/2} + z_\beta)^2 \, p(1-p)}{d^2}$

where $z_{\alpha/2}=1.96$ for 95% confidence, $z_\beta=0.8416$ for 80% power, $p$ is the expected pass rate, and $d$ is the effect size you need to reliably detect. Computing this directly, at a realistic $p=0.85$ pass rate and $d=0.01$ (a 1-percentage-point effect):

import math
z_alpha, z_beta, p, d = 1.96, 0.8416, 0.85, 0.01
n_required = 2 * (z_alpha + z_beta)**2 * p * (1-p) / d**2
print(f"{n_required:.0f}")   # 20,015

20,015 examples, confirming the real figure directly rather than citing it secondhand. This is the honest answer to “why do serious eval suites run into the thousands, not the dozens”: detecting a 1-point improvement, exactly the kind of increment real model iteration actually produces most of the time, genuinely requires an eval set on the order of 20,000 examples, not 200, and running a smaller one doesn’t make the true effect size larger, it just makes your ability to detect it worse.

Running the actual gate: a true 1% improvement, at two real sample sizes#

This is the part worth seeing executed rather than described. Set up two models with a genuinely real, small, TRUE difference: baseline pass rate 0.85, new model pass rate 0.86, an honest 1-point improvement. Sample real (simulated) eval results at $n=200$ and at $n=20{,}000$, compute a real bootstrap confidence interval on the difference, resampling 1,000 times with replacement and taking the 2.5th and 97.5th percentiles, exactly the mechanism the eval-systems process this post is built from actually specifies, and apply the real gating rule: block if the confidence interval on the difference includes zero or goes negative, pass only if it’s reliably positive.

n=   200: baseline=0.865 [0.815, 0.910]   new=0.845 [0.795, 0.895]
          observed diff = -0.020, 95% CI on diff = [-0.095, +0.050]
          CI-gate decision: BLOCK (regression or noise, can't confirm improvement)

n= 20000: baseline=0.847 [0.843, 0.853]   new=0.860 [0.856, 0.865]
          observed diff = +0.013, 95% CI on diff = [+0.006, +0.020]
          CI-gate decision: PASS (real improvement confirmed)

Look closely at the $n=200$ row: the observed difference is negative, the new model, which is genuinely 1 point better, happened to score lower than the baseline in this particular random draw, purely from sampling noise at a small sample size. A team watching only the point estimate, with no confidence interval at all, would conclude the new model regressed and block a real, genuine improvement. A team with a confidence interval, but without knowing to distrust a $\pm[-0.095,+0.050]$ interval that wide, might do the same, or worse, might see the negative point estimate, panic, and start debugging a regression that never happened. At $n=20{,}000$, the same underlying true difference is recovered accurately, $+0.013$ observed against a true $+0.01$, and the confidence interval correctly and tightly excludes zero. Same real model, same real improvement, opposite gate decision, purely as a function of how many examples the eval actually ran.

The real pipeline, put together#

The full, real promotion pipeline this post has been building toward, stated as the actual sequence a checkpoint goes through: run the benchmark suite, checking first that no example in it overlaps the training corpus at the n-gram or Min-K%-Prob level; compute the metric with a bootstrap confidence interval sized to the effect you actually need to detect, not whatever sample happened to be convenient; compare against the baseline’s own confidence interval, not just its point estimate; and block promotion specifically when the interval on the difference fails to clear zero, rather than when the point estimate alone looks bad. Skipping any one of these three steps, an uncaught contaminated example, an underpowered sample, or a point-estimate-only comparison, produces exactly the same class of failure: a launch decision made on a number that looked trustworthy and wasn’t.

Common mistakes#

Treating “the eval set doesn’t obviously overlap the training data” as proof it isn’t contaminated: Llama 2’s own real, disclosed finding, 16% of MMLU overlapping, some at over 80% token match, happened at a frontier lab running its own decontamination process, not an amateur oversight.

Assuming n-gram contamination checks catch everything: they only catch near-exact text matches, and a paraphrased leak of the same underlying content passes cleanly, which is exactly why Min-K% Prob and backdoor-based methods like DyePack exist as a second, different kind of check.

Reading a negative point-estimate difference as proof of a regression without a confidence interval: the $n=200$ run above shows a genuinely, truly improved model producing a negative observed difference from sampling noise alone, and the honest conclusion at that sample size is “cannot tell,” not “regressed.”

Sizing an eval set by convenience or existing benchmark size rather than by the effect size that actually needs detecting: a 200-example eval and a 20,000-example eval are not different quality levels of the same measurement, they’re answering different questions, and only one of them can reliably detect a real 1-point change.

Try it yourself#

Beginner. Using the ngrams function above, compute the 5-gram overlap between the sentence “the model trained on this exact sentence” and “the model trained on this exact sentence again,” and explain why a smaller n (5 instead of 13) makes the contamination check both more sensitive and more prone to false positives on generic phrasing.

Intermediate. Rerun the power calculation with $d=0.005$ (half a percentage point) instead of $d=0.01$, holding $p=0.85$ fixed. How does the required $n$ change, and why does halving the effect size you need to detect not simply double the required sample size?

Advanced. The $n=200$ bootstrap run above happened to produce a negative observed difference by chance. Design a simulation that runs the same $n=200$ experiment 500 times (with different random seeds) and reports what fraction of those runs would have incorrectly blocked the genuinely-improved model. What does that fraction tell you about the real-world reliability of shipping decisions made on a 200-example eval, beyond any single unlucky run?

What this takes to be frontier-job-ready#

OpenAI’s own Research Engineer, Frontier Evals & Environments listing states the mandate directly: “Create ambitious RL environments to push our models to their limits. Measure frontier model capabilities, skills, and behaviors… Help steer training for our largest training runs.” Its own hard requirements name “evals, graders, synthetic data” alongside RL and post-training as equally core, not a lesser specialization.

Poolside’s real posting for Member of Engineering, Evaluations Engineering makes the production-infrastructure side of this post concrete: “design a Python framework that makes it easy… to implement both internal and public benchmarks in a centralized way” and “build and maintain the pipeline that runs distributed evaluations at scale,” with a real, named tech stack, Kafka, Google Pub/Sub, GCP, AWS, Azure, Grafana, Prometheus, Datadog. That’s this entire post’s contamination-check-plus-power-calculation-plus-CI-gate pipeline, described as production infrastructure with an on-call rotation behind it, not a notebook someone runs before a launch.

---

The one-sentence version: GPT-3’s real bloom filter and 13-gram check, and Llama 2’s own disclosed finding that 16% of MMLU had leaked into its training data, are the actual, measured scale of the contamination problem this post’s pipeline exists to catch; the real power calculation behind “how big does an eval need to be” computes to 20,015 examples for a realistic 1-point effect, not a rounded rule of thumb; and running the actual bootstrap-CI gate at $n=200$ on a model with a real, true 1% improvement produced a negative observed difference and a correct “block” decision anyway, from sampling noise alone, which is the cleanest, most concrete proof available that an eval pipeline’s trustworthiness was never about which metric you picked, it was about whether the pipeline around that metric was built to actually detect the size of effect you needed it to.