Find your next learning step
A queue simulation can be watched after every batch, but an ordinary fixed-sample interval was not designed for repeated looks. Time-uniform confidence sequences protect a stated coverage guarantee across the whole monitoring path, provided the construction matches the weighted estimator and adaptation rule.
Open the chapter contents · Queueing Theory root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the decision, or read from Chapter 1. The chapters remain open; these grouped routes are a navigable index.
Route 1 · Define the evidence
Route 2 · Work the ledgers
Route 3 · Transfer the method
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE EVIDENCE
1. Repeated looks change the question
A fixed-sample interval answers: what uncertainty remains at the planned sample size? A monitored study asks a stronger question: can we look after many batches and stop whenever the evidence becomes decisive? Reusing a fixed interval at every look generally does not preserve its advertised error rate.
A confidence sequence is a sequence of intervals designed to cover the target simultaneously over time with high probability. If the sequence is valid, stopping at a data-dependent time retains the stated guarantee. The target, filtration and assumptions must be declared first.
CHAPTER 2 OF 12 · DEFINE THE EVIDENCE
2. Define the estimand and information flow
For a queue, the estimand might be the probability that virtual waiting time exceeds 90 minutes under a fixed operating policy. If importance sampling is used, each path contributes a likelihood-weighted value rather than a simple zero or one.
Record what is known before each path or batch: proposal choice, past weights, model state and stopping boundaries. An anytime claim relies on this ordered information. Future paths cannot influence past proposal choices, and a weight formula must remain valid under the actual adaptive sampling policy.
CHAPTER 3 OF 12 · WORK THE LEDGERS
3. A monitoring ledger
Consider a hypothetical rare-delay estimate checked after four cumulative batches. The confidence-sequence endpoints are illustrative outputs from a method chosen before simulation; they are not generic binomial formulas.
| Cumulative paths | Point estimate | Time-uniform interval | Decision state |
|---|---|---|---|
| 2,000 | 0.034 | 0.018–0.058 | Continue |
| 5,000 | 0.031 | 0.021–0.045 | Continue |
| 10,000 | 0.029 | 0.022–0.038 | Continue |
| 18,000 | 0.028 | 0.023–0.034 | Below 0.035 threshold |
The study stops only when the upper boundary falls below 0.035. The same rule applied to ordinary intervals recalculated repeatedly would not automatically be anytime-valid.
CHAPTER 4 OF 12 · WORK THE LEDGERS
4. Worked decision boundaries
Suppose the operational rule has three zones: approve capacity if the upper boundary is below 0.035, escalate if the lower boundary is above 0.050, and otherwise continue until a resource cap. This makes inconclusive evidence a legitimate outcome.
| Look | Lower bound | Upper bound | Rule result |
|---|---|---|---|
| 1 | 0.018 | 0.058 | Neither boundary crossed |
| 2 | 0.021 | 0.045 | Neither boundary crossed |
| 3 | 0.022 | 0.038 | Neither boundary crossed |
| 4 | 0.023 | 0.034 | Approve under declared rule |
A point estimate of 0.034 at the first look is below the target but does not trigger approval because the upper uncertainty boundary remains 0.058.
CHAPTER 5 OF 12 · WORK THE LEDGERS
5. Anytime validity has a price
Time-uniform intervals are often wider than a fixed-sample interval at one planned time because they protect against all monitored times. The width is not wasted precision; it buys flexibility to continue, stop or extend the study without a hidden peeking penalty.
Choose the method for the variable actually observed. Bounded path contributions allow different tools from heavy-tailed importance weights. Truncating or smoothing weights changes the estimand or introduces bias unless the correction is included transparently.
CHAPTER 6 OF 12 · TRANSFER THE METHOD
6. Connect adaptation and martingale logic
Nonnegative supermartingales and betting constructions are common foundations for time-uniform bounds. Their conditional expectation requirements must hold under the declared filtration. Adaptive proposals can be allowed only when the likelihood-weighted increment remains correctly centred or otherwise satisfies the chosen process conditions.
Do not paste a confidence sequence for independent Bernoulli data onto adaptive, weighted paths. If the proof does not cover the estimator, freeze the proposal and sample size or use a method developed for the relevant adaptive setting.
CHAPTER 7 OF 12 · TRANSFER THE METHOD
7. Changed-context transfer: ambulance queues
Transfer the design to an ambulance-dispatch simulation monitoring the probability that response delay exceeds a service threshold. Predeclare an approve boundary, an escalation boundary and a maximum computational budget.
The transfer check asks whether simulations are conditionally generated as assumed, whether scenario weights are bounded enough for the selected sequence and whether policy changes during monitoring create a new estimand. Statistical validity does not choose the public-service threshold.
CHAPTER 8 OF 12 · TRANSFER THE METHOD
8. Connect the rare-delay branch
The sequential budget guide separates efficiency stopping from evidence stopping. Pilot allocation explains how shares are selected before production.
Use tail-weight diagnostics before trusting a narrow interval; one dominant path can expose a mismatch between the method and the evidence.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
Anytime-valid does not mean assumption-free, unbiased under every adaptation or valid for every post hoc threshold. Selecting the delay threshold after watching the path introduces another layer of multiplicity unless the procedure covers that selection.
A confidence sequence can remain wide forever when the event is too rare, weights are unstable or the decision margin is small. The correct conclusion may be that the available simulation budget cannot separate the operational choices.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Practical evidence checklist
Archive the estimand, queue policy, filtration, path generator, proposal probabilities, weights, batch times, confidence-sequence formula, coverage level, decision thresholds and resource cap. Record every look, including those that did not support publication or action.
Check simulated calibration, largest weights, effective counts, replicate paths and the conclusion under a fixed-sample analysis at the final time. Explain any weight clipping and report the resulting bias sensitivity separately.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Can I stop when the point estimate crosses the target?
Not with the stated uncertainty guarantee. Use the predeclared interval boundary or another anytime-valid decision rule.
Must I inspect after every path?
No. A time-uniform sequence permits flexible looks; operationally, batches can reduce overhead and make the information flow easier to audit.
Does a confidence sequence solve proposal instability?
No. It needs increments that satisfy its assumptions. Proposal diagnostics and support checks remain essential.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Sources and next reading
Howard, Ramdas, McAuliffe and Sekhon develop time-uniform nonparametric confidence sequences. Waudby-Smith and Ramdas study betting-based sequential confidence intervals for bounded means. Waudby-Smith and colleagues discuss time-uniform central-limit theory and asymptotic confidence sequences. The queue ledgers above are original teaching examples.
Continue through the Queueing Theory root, the Gold Standard of Education and How X Works.
Top · Contents · Previous · How X Works