Loading…
Reading the draws and working out what chance predicts.
Forecast Arena
Five complete probability models forecast each combination using only earlier draws. The first 70% may choose a challenger; its identity is then locked before the final 30% is scored against the fair machine.
Cash Pop1/15 era · since Apr 2023420 draws
Ville’s inequality caps the chance that a valid e-value EVER reaches 20 at five per cent, however many times you look. This one is at 5.56e-3.
The largest of the four is not a valid reading. Each system’s e-value is a test martingale on its own, but picking the winner after seeing the scores is a selection, and Ville’s inequality says nothing about the maximum of several martingales. A convex mixture of martingales is one, which is why the mean is the headline and the four figures below it are descriptive.
Exact weighted-subset prequential scoringprequential-weighted-subset-v2full-combination log score; selection uses discovery only
Cash Pop1/15 era · since Apr 2023420 draws
Cash Pop1/15 era · since Apr 2023180 draws
Exact weighted-subset prequential scoringprequential-weighted-subset-v2full-combination log score; selection uses discovery only
Each challenger assigns positive weights . Its probability for a complete -set is
where sums the weight-products of every possible -set. Inclusion probabilities are derived from that same distribution, so they always lie between zero and one and sum to .
At draw , weights may use draws but never draw . The log evidence against the fair machine is
The winning challenger is chosen using discovery data only. Its confirmation likelihood ratio is calculated exclusively from later draws, preserving a genuine test boundary.
is a test martingale against the fair machine: its expectation under the null is one at every round, because the weights at draw are a function of alone. Ville's inequality then gives
so an e-value that ever reaches 20 is evidence at five per cent — and it may be looked at after every single round without spending anything, which no p-value can survive.
That guarantee belongs to one martingale. The largest of four is not one: choosing the winner after seeing the scores is a selection, and Ville says nothing about a maximum. But a convex mixture of martingales is a martingale, so
with the prior of a quarter each declared before any draw was scored, is a single valid e-value for the compound question does any of these four beat the fair machine. It is shown first, and the four individual figures below it are descriptive. Because a mean never exceeds a maximum, the mixture is always the more conservative of the two — which is the price of having looked at all four.
A description can fit history beautifully while knowing nothing about the future. Proper scoring rules force a model to state how probable every outcome was before it happened; confident mistakes cost more than cautious ones.
The Vault makes the social rule visible as well as mathematical: exploration is welcome on one side of the seal, but only untouched later evidence may confirm the selected idea. This follows the logic of strictly proper scoring rules.