Find your next learning step
A pilot can reveal that one proposal is wasting paths, but moving the remaining budget is a statistical decision. The safe workflow records what was known at each batch, protects every relevant tail route and separates an efficiency stop from an evidence claim.
Open the chapter contents · Queueing Theory root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the decision, or read from Chapter 1. The chapters remain open; these grouped routes are a navigable index.
Route 1 · Define the evidence
Route 2 · Work the ledgers
Route 3 · Transfer the method
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE EVIDENCE
1. Reallocation changes the experiment
Suppose three queue proposals target arrival surges, long services and recovery failures. After a pilot, the recovery proposal may carry more useful weight for a 90-minute delay. Moving paths toward it can lower variance, yet the move was learned from random pilot outcomes. The final estimator and uncertainty calculation must match that learning rule.
A sequential budget is not one invisible pool. Treat it as dated batches: observe diagnostics, apply a prespecified rule, record the next allocation, then draw new paths. This filtration prevents later evidence from being written back into an earlier decision.
CHAPTER 2 OF 12 · DEFINE THE EVIDENCE
2. Define pilot regret before chasing the winner
Pilot regret is the efficiency lost because the chosen share vector differs from the unknown best allocation. A noisy pilot can overfund a proposal that happened to look stable or abandon a route that produced no hits by chance.
Use regret as a planning quantity: compare predicted variance or decision loss under the chosen shares with credible alternatives, while maintaining support floors. It does not excuse invalid weights, missing mechanisms or an optional stop selected for a favourable estimate.
CHAPTER 3 OF 12 · WORK THE LEDGERS
3. A three-batch allocation ledger
A hypothetical study has 12,000 path-equivalents after a 1,800-path pilot. Each row records the rule available before the next batch begins.
| Stage | New paths | Arrival / Service / Recovery | Decision made before draw |
|---|---|---|---|
| Pilot | 1,800 | 33% / 33% / 34% | Fixed start |
| Production 1 | 4,000 | 20% / 35% / 45% | Pilot rule |
| Production 2 | 4,000 | 15% / 30% / 55% | Batch-1 diagnostics |
| Audit | 2,200 | 25% / 35% / 40% | Frozen independent check |
The audit deliberately backs away from the apparent winner. That costs some efficiency but reveals whether the production concentration depended too heavily on one pilot seed.
CHAPTER 4 OF 12 · WORK THE LEDGERS
4. Worked regret and threshold coverage
Assume the selected production shares predict relative variance 1.00 at 60 minutes, 1.25 at 90 minutes and 1.80 at 120 minutes. A balanced alternative predicts 1.10, 1.20 and 1.45. If the decision protects all three thresholds, the selected rule has maximum relative variance 1.80 rather than 1.45: a worst-threshold regret ratio of about 1.24.
| Threshold | Selected rule | Balanced rule | Selected ÷ balanced |
|---|---|---|---|
| 60 min | 1.00 | 1.10 | 0.91 |
| 90 min | 1.25 | 1.20 | 1.04 |
| 120 min | 1.80 | 1.45 | 1.24 |
| Maximum | 1.80 | 1.45 | 1.24 |
A minimax threshold rule would prefer the smaller maximum; a policy-weighted rule may differ. State the loss before examining the final estimates.
CHAPTER 5 OF 12 · WORK THE LEDGERS
5. Use coverage floors and spending caps
Every proposal needed for support or a plausible overload mechanism receives a floor. A cap prevents a volatile pilot component from consuming the whole budget. Allocate only the remainder with the adaptive score.
Floors are not decorative equal shares. Link each floor to support, a threshold or an audit requirement. If floors consume nearly all available paths, the proposal family is too ambitious for the budget and should be simplified rather than silently underpowered.
CHAPTER 6 OF 12 · TRANSFER THE METHOD
6. Safe stopping is two different questions
An efficiency stop asks whether another batch is unlikely to change allocation enough to justify its cost. An inferential stop asks whether the evidence is sufficient for a claim. The first can use stable shares; the second needs a confidence sequence, an alpha-spending design or another method valid under the declared stopping rule.
A conventional fixed-sample interval recalculated after every batch is not automatically safe to stop on. Optional stopping can favour the first attractive fluctuation. If no anytime-valid method is used, freeze the analysis sample size before viewing the final decision statistic.
CHAPTER 7 OF 12 · TRANSFER THE METHOD
7. Changed-context transfer: emergency call queues
Transfer the method to an emergency call network with surge, long-handle and dispatch-feedback proposals. Reallocate only after a batch closes, retain a baseline defensive share and report each service threshold separately.
The transfer check asks whether proposal likelihoods cover the full dependent path and whether the staffing consequence is represented in the loss. Monte Carlo precision cannot decide ethical service priorities or repair a queue model that omits callback behaviour.
CHAPTER 8 OF 12 · TRANSFER THE METHOD
8. Connect the allocation branch
Start with pilot allocation and variance-aware shares. The multiple-proposal MIS guide explains combination weights, while defensive mixtures protect tail coverage.
After reallocation, inspect tail-weight diagnostics; a large nominal batch with a few dominant weights is not strong evidence.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
Stable allocations do not prove stable answers. Two share vectors can look similar while the rarest threshold is controlled by different paths. Pilot regret estimates are themselves noisy, and repeated optimisation can overfit the diagnostic sample.
Stopping because the point estimate crossed a capacity target is not the same as stopping because uncertainty is narrow enough. More adaptive stages also do not repair missing support, reversed likelihood ratios or an incorrect queue transition model.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Evidence and reproducibility checks
Archive seeds, proposal versions, log densities, per-path costs, batch boundaries, diagnostics available at each decision, floors, caps and the exact allocation function. Keep threshold definitions and operational loss fixed before final analysis.
Report predicted versus realised variance, effective proposal counts, maximum weights, replicate estimates and conclusions under nearby share vectors. Re-run the audit with its frozen allocation; disagreement is evidence to investigate, not an inconvenient replicate to pool away.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
May the shares change after every path?
Only with an estimator and proof that cover that policy. Batches are easier to audit because the information set and next action are visible.
When is the pilot large enough?
When it can distinguish allocations at the decision-relevant thresholds with acceptable regret, not merely when every proposal records a hit.
Can a confidence sequence replace model checking?
No. It handles sequential uncertainty under its assumptions; it does not validate the proposal laws or the queue model.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Sources and next reading
He and Owen develop sequential estimation of efficient MIS mixture weights. Portier and Delyon study adaptive importance sampling across stages. Delyon and Portier analyse safe adaptive mixtures with a defensive component. The ledgers above are original teaching examples.
Continue through the Queueing Theory root, the Gold Standard of Education and How X Works.
Top · Contents · Previous · How X Works