GOLD STANDARD · PROBABILITY · HOW X WORKS
Find the decision hidden inside the model
An off-policy estimate can look favourable at one checkpoint and reverse after more adaptive review data arrive. Confidence sequences keep repeated monitoring honest, while logging-drift diagnostics and deployment gates stop a zero-truncated GIGP review policy from being promoted beyond the evidence that supports it.
This guide connects the Gold Standard of Probability, Gold Standard of Education, and How X Works hub.
Chapter contents and reading routes
Choose the route closest to your present task. Every chapter remains expanded, so the article also reads naturally from start to finish.
Orient anytime-valid evaluation
Build the confidence sequence
Diagnose logging drift
Transfer and govern
Use evidence and next routes
CHAPTER 1 OF 12 · ORIENT THE DECISION
1. Repeated checking changes the probability problem
A team may inspect an estimated policy gain every day and stop when the result looks convincing. An ordinary fixed-sample interval is not designed for that stopping rule; repeated peeking can make an unhelpful policy appear successful more often than its nominal error rate suggests.
A confidence sequence is a sequence of intervals intended to cover the target simultaneously across time under stated assumptions. It lets an evaluator stop, continue or wait for budget without pretending the stopping time was fixed in advance.
CHAPTER 2 OF 12 · ORIENT THE DECISION
2. Name the policy value and the safety constraints
For active review, value might be reduction in population-total loss per review hour. The outcome must connect the reviewed labels to the zero-truncated generalised inverse Gaussian–Poisson estimate rather than reward only easy classifications.
Write separate gates for average value, specialist hours, high-consequence false links and minimum subgroup coverage. A positive mean gain does not compensate for violating a hard review-capacity or fairness condition.
CHAPTER 3 OF 12 · BUILD ANYTIME-VALID EVIDENCE
3. Build the off-policy score from logged propensities
At each decision, preserve the eligible actions, historical context, logging probability, chosen action, review cost and later reward. An evaluation score reweights an observed reward when the candidate policy agrees with the logged action.
If the candidate chooses an action with logging probability zero, its value is not identified by that log. A direct reward model may extrapolate, but it does not manufacture experimental support; the unsupported region belongs outside the deployment gate.
CHAPTER 4 OF 12 · BUILD ANYTIME-VALID EVIDENCE
4. Work through an anytime-valid comparison
Suppose the estimated candidate advantage after 200 reviews is 4.0 decision units, with a confidence sequence from -1.5 to 9.5. After 500 reviews the estimate is 3.4 with interval 0.4 to 6.4, and after 800 it is 3.1 with interval 1.1 to 5.1.
The sequence first crosses a zero-gain gate at 500 reviews and remains above it at 800. Promotion still waits if the budget or subgroup sequences have not crossed their own safe boundaries. One encouraging checkpoint is evidence, not the whole decision.
| Reviews | Estimated gain | Confidence sequence | Gate status |
|---|---|---|---|
| 200 | 4.0 | -1.5 to 9.5 | Continue |
| 500 | 3.4 | 0.4 to 6.4 | Value gate crossed |
| 800 | 3.1 | 1.1 to 5.1 | Value remains supported |
CHAPTER 5 OF 12 · BUILD ANYTIME-VALID EVIDENCE
5. Use variance controls without hiding fragility
Inverse propensities can create large contributions when the logger rarely chose an action. Self-normalisation, reward bounds, doubly robust scores and predictable clipping can stabilise a sequence, but each changes its assumptions or target.
Show maximum weight, effective sample size and sensitivity to caps beside the interval. If the lower bound becomes positive only after severe clipping, collect more exploration rather than treating the stabilised result as decisive.
CHAPTER 6 OF 12 · DIAGNOSE DRIFT
6. Detect adaptive logging drift in the data actually used
A logger may change because reviewers learn, capacity shifts, a new feature is introduced or a previous policy version is retired. Compare recorded propensities, eligible action sets, context distributions and realised action frequencies by version and time.
Recompute propensities only when the original decision rule is recoverable. A score saved after human overrides is not necessarily the probability that generated the action. Unknown logging mechanisms require a sensitivity analysis or a new prospective logging period.
CHAPTER 7 OF 12 · DIAGNOSE DRIFT
7. Separate harmless mixture change from support loss
More difficult cases may arrive without invalidating conditional rewards if all relevant actions remain represented. Support loss occurs when a candidate relies on contexts or actions that the logger almost never explored.
Suppose a specialist action falls from 12% to 2% of eligible assignments and its minimum propensity falls from 0.08 to 0.005. The average context shift may look modest, yet off-policy variance and extrapolation risk rise sharply. Pause promotion for that region even if overall value remains positive.
| Diagnostic | Stable signal | Stop signal |
|---|---|---|
| Minimum propensity | Above declared floor | Near zero |
| Effective sample size | Grows with log | Stalls despite more records |
| Policy-version balance | Both policies represented | One version dominates |
| Subgroup support | Gate met in every critical group | Unsupported critical group |
CHAPTER 8 OF 12 · TRANSFER AND GOVERN
8. Changed-context transfer: adaptive quality inspection
Move from record-linkage review to factory inspection. The stable structure is an adaptive logger, costly labels, counterfactual policy value and repeated monitoring. Defect consequences, action eligibility and production feedback change.
Transfer succeeds when the reward, exposure unit and interference path are rebuilt. A review action in one batch can alter later production, violating the no-interference assumption that may have been plausible in a static linkage queue.
CHAPTER 9 OF 12 · TRANSFER AND GOVERN
9. Build deployment as staged evidence, not one switch
Start with shadow evaluation, then a guarded fraction of supported traffic, then broader rollout only after value and safety sequences remain inside their gates. Preserve a conservative fallback and version every change.
A deployment gate should specify entry, pause and rollback rules. Hysteresis can prevent oscillation: promotion may require a strong lower bound, while rollback can occur sooner when a safety upper bound crosses its limit.
CHAPTER 10 OF 12 · TRANSFER AND GOVERN
10. Limits and common misconceptions
Anytime-valid does not mean assumption-free. Logged propensities, reward bounds or moment conditions, measured context and stable target definitions still matter. Confidence sequences can be wide when actions are rare, which is information about the design rather than a defect to conceal.
A drifting logger is not automatically invalid, and a stationary logger is not automatically safe. The relevant question is whether the score construction remains predictable and whether every candidate action has credible support where deployment is proposed.
CHAPTER 11 OF 12 · CHECK AND CONTINUE
11. Questions evaluation teams often ask
Use a fixed-sample interval when the sample size and analysis time truly were fixed. Use an anytime-valid method when data-dependent stopping, continuous dashboards or repeated promotion decisions are part of the workflow.
Can the policy deploy as soon as the value bound is positive?
Only if every prespecified capacity, subgroup and harm gate is also satisfied and the supported deployment region matches the evaluation data.
Should old logs be discarded after drift?
Not automatically. Stratify or model policy versions when their propensities and contexts are recoverable, and exclude only records whose generating mechanism cannot support the target evaluation.
CHAPTER 12 OF 12 · CHECK AND CONTINUE
12. Evidence, specialist routes and the next check
off-policy confidence-sequence research develops time-uniform evaluation for adaptively collected contextual-bandit data. high-confidence off-policy evaluation frames conservative policy comparison, while safe policy improvement with baseline bootstrapping shows how support can constrain deployment.
Continue through off-policy evaluation and review-budget safety, anytime-valid audit stopping, and reviewer drift and policy change. The next check is to place value, support, budget and subgroup sequences on one versioned deployment record.
Contents · Previous chapter · Return to the Probability root