Find your next learning step
A restored comparator becomes useful only after it can run reliably and its disagreements can be interpreted. The next task is to collect shadow evidence without letting either rule quietly influence the other. Discordance must then be separated into candidate drift, comparator drift, population shift, measurement error and common-mode failure. Re-expansion is justified only when the relevant gate is met with protected evidence, working fallbacks and a decision boundary that matches the observed queue.
Open the chapter contents · Queueing Theory root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the decision, or read from Chapter 1. The chapters remain open; these grouped routes are a navigable index.
Route 1 · Define the decision
Route 2 · Work the ledgers
Route 3 · Transfer the method
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE DECISION
1. Define the shadow-evidence claim
Freeze the candidate, restored comparator, arrival and service definitions, class map, data contract, authority cap, excluded strata and observation window. State what a disagreement means operationally before opening outcomes.
Shadow operation means the comparator records a recommendation but cannot route, prioritise or delay a case. Log both outputs against the same decision-time snapshot so later differences are not created by unequal information.
CHAPTER 2 OF 12 · DEFINE THE DECISION
2. Separate disagreement from decision error
A discordance is simply two different outputs. It becomes decision evidence only after the actual state, operational consequence and relevant limit are known. Agreement is also not proof of correctness when both rules inherit the same faulty field.
Use a three-way review: candidate better supported, comparator better supported or unresolved. This prevents a familiar rule from winning merely because it is older.
CHAPTER 3 OF 12 · WORK THE LEDGERS
3. Work a protected shadow register
A hospital flow team observes a restored comparator for 1,200 eligible daytime arrivals while the candidate remains capped at 20% automatic assignment.
| Stratum | Cases | Discordant | Reviewed | Comparator supported | Candidate supported | Unresolved |
|---|---|---|---|---|---|---|
| Ordinary medical | 620 | 37 | 37 | 8 | 24 | 5 |
| Ordinary surgical | 360 | 31 | 31 | 15 | 11 | 5 |
| Transfer arrivals | 140 | 28 | 28 | 19 | 4 | 5 |
| High surge | 80 | 21 | 21 | 8 | 6 | 7 |
The overall disagreement rate is 9.8%, but the decision turns on mechanism and stratum. Transfer arrivals remain outside expansion while ordinary medical cases may proceed to a separate gate.
CHAPTER 4 OF 12 · WORK THE LEDGERS
4. Triage discordance by mechanism
For every reviewed case, test candidate inputs, comparator inputs, label timing, class mapping, service estimates and downstream outcome. Assign a provisional mechanism only when the trace supports it.
| Mechanism | Diagnostic clue | Immediate control | Re-expansion effect |
|---|---|---|---|
| Candidate drift | Error rises after candidate update | Rollback or narrow version | Stop |
| Comparator drift | Stale service table | Repair comparator | Do not count window |
| Population shift | New transfer pattern | Create new stratum | Exclude until qualified |
| Measurement error | Outcome timestamp mismatch | Correct data pipeline | Repeat evidence |
| Common-mode failure | Both share wrong bed-state feed | Restore independent check | Stop |
Keep unresolved cases visible. Forcing uncertain cases into a winner category makes the gate look cleaner than the evidence.
CHAPTER 5 OF 12 · WORK THE LEDGERS
5. Set re-expansion gates before review
A gate can require zero unsafe reversals, bounded override and tail-wait differences, minimum stratum exposure, successful fallback drills, complete version linkage and a maximum unresolved-discordance rate.
Apply conjunction for necessary conditions: one failed safety or lineage gate blocks expansion even when averages improve. Separate desirable benefits from minimum protections.
CHAPTER 6 OF 12 · TRANSFER THE METHOD
6. Protect the evidence from adaptive reuse
Use the first shadow window to diagnose and repair. Once the team changes a threshold, class map or service estimate, close that window and open a later protected confirmation stream.
Record stopping rules and interim looks. NASA’s model-and-simulation standard emphasises defined acceptance criteria, input pedigree, uncertainty records and credibility tied to a particular use.
CHAPTER 7 OF 12 · TRANSFER THE METHOD
7. Changed-context transfer: airport stand assignment
Transfer the method to an airport whose candidate rule assigns ordinary stands while a restored legacy rule runs in shadow. Discordance clusters during towing transitions rather than ordinary arrivals.
The airport isolates towing transitions, repairs a stale turnaround input and confirms ordinary arrivals on a new week. It expands only the ordinary-arrival cap; towing cases remain dispatch-reviewed.
CHAPTER 8 OF 12 · TRANSFER THE METHOD
8. Connect restoration to earlier authority decisions
Begin with expansion-stage monitoring, withdrawal-reversal triggers and restored-comparator requalification. Use bounded-authority expansion, comparator withdrawal and renewal-failure rollback for the preceding authority state.
Return to post-commissioning drift and successor requalification when evidence supports a repaired successor rather than re-expanding the current candidate.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
Low discordance does not show that either rule is good; both may share a blind spot. High discordance does not prove the candidate failed; the comparator or population may have changed.
Shadow evidence does not reproduce live feedback effects. A rule that changes priorities can alter later queue states, so a bounded live stage is still needed after shadow qualification.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Practical evidence and gate checks
Retain rule versions, input snapshots, matched outputs, sampling frame, review records, mechanism codes, protected-window boundaries, limits, fallback drills and authority decisions. Link every case to its queue state at decision time.
Before expansion, verify that excluded strata cannot enter automation, alerts reach an owner and the previous cap can be restored promptly. Monitor the first expanded window as a new claim.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Must every discordant case be reviewed?
Not always. Use complete review for rare high-consequence cases and a prespecified sample for common low-consequence cases, with enough coverage for each gate.
Can agreement count toward confirmation?
Yes, but only alongside independent outcome and integrity checks. Agreement alone cannot reveal common-mode failure.
Can one stratum expand while another stays capped?
Yes. Stratum-specific authority is often the clearest match to uneven evidence.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Sources and next reading
NASA’s Standard for Models and Simulations requires acceptance criteria, input pedigree and credibility assessment for the intended use. The NASA-HDBK-7009B implementation handbook adds current guidance on risk, caveats and credibility reporting. The hospital and airport cases are original teaching examples.
Continue through the Queueing Theory root, the Gold Standard of Education and How X Works.
Top · Contents · Previous · How X Works