Every machine, every set of balls, tested on its own

Testing a set of balls rather than an era

Per set. For one ball set drawn $D$ times from a pool of $N$, taking $k$ numbers each time, every number is expected $Dk/N$ times. The statistic is the usual one,

$X^2=\sum_{i=1}^{N}\frac{(O_i-E)^2}{E},\qquad E=\frac{Dk}{N},$

but its null distribution is not $\chi^2_{N-1}$, because the $k$ numbers of a draw are taken without replacement and are therefore negatively correlated. Joe (1993) gives the correction: multiply by

$\frac{N-1}{N-k}$

before referring to $\chi^2_{N-1}$. On a 6-from-59 game that is a factor of 1.09 — small, and enough to move a borderline p by a third.

The reading is two-sided. A fit too good is as suspicious as one too bad; it is the signature of fabricated data, and it is how several scientific frauds were caught.

The multiplicity is exact here. Eleven sets tested at 5% each would flag one somewhere more often than not, so every p is multiplied by the number of sets tested. This is one of the few places on the site where a Bonferroni correction is not conservative: the operator chose the sets, they were fixed before any of this was read, and there is no hidden search over which ones to look at.

Across sets. A contingency table of numbers against sets asks a different question — not "is this set fair" but "do the sets differ". The statistic is the usual contingency $X^2$ with $(N-1)(S-1)$ degrees of freedom and the same rescaling, and it is reported beside a permutation p: shuffle which set each draw belonged to, keeping the set sizes fixed, and recount. Under the null that the numbers are unrelated to the set, that shuffle is the null exactly, with no distributional assumption at all. When the two disagree, the approximation is what to distrust.

Anytime-valid, per set. Each set also carries a Beta(½,½)-mixture e-process, which can be looked at after every draw without inflating anything — the property a p-value does not have. An e-value of 20 is the standing alarm level; the values here run below one, which is evidence for fairness rather than merely an absence of evidence against it.

What this cannot see. A ball drawn 15% more often than its siblings needs roughly 2,000 draws of its own set before this test would notice. The busiest set in this data has a few hundred. That is not a defect to hide — it is the sensitivity of the instrument, and the page states it beside the result, because "nothing found" means "nothing this large" and nothing more.

Where a fault would actually be

If a lottery were unfair by accident, the fault would be a ball.

Not a conspiracy, not a rigged computer — a sphere of injection-moulded rubber a fraction of a gram heavier than the forty-eight beside it, or one that has picked up a scuff over three hundred tumbles. Such a ball would rise or fall a little differently in the air, and over enough draws it would come out slightly more or slightly less often than its siblings.

Here is the thing that makes it hard to find: that ball belongs to one set.

The UK's national game does not draw with a single set of balls. It rotates four machines with names — Arthur, Guinevere, Lancelot, Merlin — across a dozen numbered sets, and it publishes which pair drew every result. So a faulty ball appears in maybe one draw in ten. Pool a whole era of results and you have averaged it together with eleven sets that are fine, which is exactly how you would design a test to miss it. John Haigh made this point about the UK game in 1997, and it remains the reason this page exists.

The other reason is the Triple Six Fix. In April 1980 the Pennsylvania Lottery's Daily Number came up 666, and it did so because Nick Perry had injected latex paint into every ball except the fours and the sixes. That is what a rigged draw looks like: one set of balls, one machine, one night, and a payout that broke the game. It was caught by bookmakers noticing the money, not by statistics — but it is the shape of fault this instrument is built for.

So: five games on this site publish their machine and their ball set, and every set with enough draws is tested on its own here, with the multiplicity corrected because testing eleven sets at five per cent each would manufacture a scandal about once a fortnight.

The result is that nothing is out of place. Every set of UK Lotto, Thunderball, Set For Life and EuroMillions sits comfortably inside what chance allows; the sets do not differ from one another; and the anytime-valid evidence runs below one, which is a mild vote in favour of the machines rather than a shrug.

That is a boring finding, and it is the correct one. It is also worth saying exactly how boring: this test would not notice a ball coming up fifteen per cent too often until that set had been drawn about two thousand times, and the busiest set here has a few hundred. What the page can honestly report is that there is no large fault, in the only place a fault could be.