A fourth thaw is not permission to search every threshold, feature and routing rule until one historical result looks attractive. It is a bounded response to a named failure or unsupported operating scope discovered after the third re-freeze. The first task is therefore to define what may change and what must remain fixed.
This guide continues the release-monitoring work. It shows how to turn a fourth-thaw gate into a small portfolio of mechanism-linked challengers, remove candidates for declared reasons, protect capacity while comparisons run and re-freeze only when the surviving rule transfers beyond the window that produced it.
The Gold Standard is restraint with evidence. A portfolio is useful because several plausible mechanisms can be compared under one plan. It becomes dangerous when pruning follows whichever candidate has the friendliest outcome after repeated inspection.
A practical orientation
Keep five objects separate: the frozen reference, the affected population, the suspected mechanism, the challenger portfolio and the protected service consequences. A candidate may improve assignment fit yet fail because it sends unsustainable work to the receiving queue.
The worked example starts with four challengers after a new referral source enters the system. Two are removed before outcome comparison because they do not address the declared mechanism or breach the capacity envelope. The transfer check moves the method to a multilingual intake stream.
- Bound the fourth thaw — for owners defining what may change.
- Build and prune challengers — for analysts avoiding outcome fishing.
- Follow the four-candidate example — for an end-to-end comparison.
- Apply fourth re-freeze gates — for accountable release decisions.
- Transfer to another channel — for changed information and demand.
Chapter contents
Open a route that matches the decision in front of you, or read straight through. Each route returns to the same evidence chain.
Start with the operating question
Build the evidence chain
Predeclare pruning reasons before looking at winners
Work the example
1. Translate the fourth-thaw signal into a bounded question
Write the signal as a testable operating claim: for which arrivals, channel, information state and capacity condition does the third re-freeze no longer have adequate support? Name the consequence that matters and the observation that could show the signal was only ordinary variation.
Freeze everything outside that question. If the signal concerns low-information referrals, do not simultaneously change ordinary-channel thresholds, service-time estimates and priority logic. A narrow thaw preserves attribution and keeps the last verified rule available as a safe reference.
- Affected population and explicit exclusions.
- Suspected mechanism and competing explanations.
- Fields, boundary components or routes allowed to change.
- Capacity and protected-outcome constraints.
- Stopping, rollback and fourth re-freeze authority.
2. Turn mechanisms into a small challenger portfolio
Each challenger should express a distinct explanation, not a cosmetic parameter sweep. One might add a missingness-aware route, another recalibrate within the affected source and a third retain the boundary but alter the receiving-service rule. Include the frozen reference and, where useful, a deliberately simple baseline.
Document the decision-time inputs, expected changed set, donated and received work, failure mode and reason for inclusion before replaying outcomes. This makes the portfolio auditable and prevents later candidates from being invented around the observed result.
| Candidate | Mechanism claim | Expected evidence |
|---|---|---|
| Frozen reference | No structural rule failure | Signal disappears after composition and capacity adjustment |
| Missingness-aware route | New source lacks a decision-time field | Movement concentrates in incomplete records |
| Source calibration | Comparable source-specific risk shifted | Within-source replay improves without leakage |
| Capacity guard | Receiving work, not assignment fit, causes failure | Protected tail recovers under bounded load |
3. Predeclare pruning reasons before looking at winners
Prune a challenger when it violates scope, uses information unavailable at decision time, fails an implementation check, exceeds the approved capacity envelope or contradicts the mechanism it was meant to test. Those are eligibility failures, not disappointing outcomes.
Do not prune merely because an early estimate is less favourable, then continue observing the remaining candidates until one crosses a convenient line. If sequential decisions are planned, declare the evidence windows, error control and action attached to each review.
4. Use common records, clocks and capacity assumptions
Replay every candidate on the same eligible decision-time records. Preserve missingness, arrival order, reason codes and the information actually available at assignment. A challenger evaluated on a cleaner retrospective dataset is answering an easier question.
Translate changed assignments into service demand with a common clock and service-time model, then test plausible stress states. A candidate that moves fewer cases can still create more work if those cases are complex or arrive together.
| Layer | Common basis | Failure if misaligned |
|---|---|---|
| Population | Same eligibility and exclusions | Composition masquerades as challenger benefit |
| Information | Same decision-time fields | Retrospective leakage |
| Time | Same arrival and outcome clocks | Delay comparisons lose meaning |
| Capacity | Same ordinary and stress envelopes | Paper gains fail in service |
5. Separate screening evidence from confirmation evidence
Use one evidence portion to remove infeasible candidates and understand mechanisms, then reserve another portion for the final comparison. The confirmation set should include the operating cycles and subgroups named in the thaw plan, not simply a random remainder that misses the relevant stress state.
A cumulative-sum or other sequential monitor can be useful for small sustained shifts, but its reference and response rule must be declared. It does not repair repeated comparison, changing denominators or post-hoc subgroup selection.
6. Worked example: four challengers for a new referral source
A new partner source doubles the low-information share and raises the specialist tail. The portfolio contains the frozen reference, C1 adding a missingness-aware parent route, C2 lowering the global threshold, C3 using a field populated only after assignment and C4 keeping assignments but adding a capacity guard.
C3 is removed for decision-time leakage. C2 is removed because its global threshold changes supported channels and breaches the specialist-work ceiling; it does not match the source-specific mechanism. C1 and C4 continue beside the reference. On held-out cycles, C1 reduces unsupported specialist assignments while C4 protects the tail during demand bursts. A combined, pre-specified C1-plus-C4 candidate is then tested as a new challenger rather than declared the winner from the same records.
| Candidate | Pruning or test result | Decision |
|---|---|---|
| Frozen reference | Valid benchmark; tail breach persists in new source | Retain for comparison |
| C1 missingness-aware route | Mechanism matched; inside work ceiling | Advance |
| C2 global threshold | Out-of-scope movement and capacity breach | Prune |
| C3 later field | Information unavailable at decision time | Prune |
| C4 capacity guard | Protects burst tail; no assignment-fit claim | Advance |
7. Test the surviving challenger in shadow and limited release
Run the final candidate in shadow on current arrivals, then release to a bounded population with the frozen reference still executable. Compare changed cases, donated and received work, delay, protected outcomes and reason-code integrity. Keep ordinary and stress results separate.
The limited release should contain a stopping rule for harm, a restriction rule for unsupported channels and a rollback package that has been exercised. A promising average is not a substitute for a controllable operating state.
8. Set fourth re-freeze gates around the whole system
Fourth re-freeze means that a tested challenger becomes the reference for a defined envelope. Require mechanism support, changed-set stability, capacity feasibility, protected-consequence acceptance, reproducibility and independent review of the evidence path.
State what remains outside the claim. If the trial covers one referral source under ordinary staffing, the re-freeze does not automatically extend to every partner, language, campaign or stress level.
- Candidate and transformation versions fixed.
- Held-out and limited-release results meet declared gates.
- Receiving and displaced queues remain inside service limits.
- Protected outcomes and subgroup uncertainty are acceptable.
- Rollback, monitoring and future-thaw triggers are owned.
9. Choose re-freeze, restrict, extend or return to the reference
Re-freeze when the candidate passes for the named envelope. Restrict when one source or stress state remains unsupported. Extend the trial only when a named mechanism has not yet been exercised and the safe route remains credible. Return to the reference when harm, leakage or uncontrollable work appears.
An extension needs a new end condition. Repeatedly adding data until a preferred candidate passes turns uncertainty into a moving target and should be recorded as a new study, not a continuation of the old confirmation test.
10. Keep the portfolio, evidence ledger and negative results
Preserve why each challenger existed, which records it used, when it was pruned and what evidence ruled it out. Negative results prevent future teams from rebuilding the same unsafe candidate under a new label.
Also retain the frozen reference outputs and the exact re-freeze decision. Monitoring after release should be able to reconstruct whether later drift comes from population, implementation, capacity or the challenger itself.
11. Misconceptions and practical questions
A portfolio is not permission for unlimited experimentation, pruning is not the same as selecting the highest estimate and a re-freeze is not proof of universal superiority. The claim is always tied to a population, information state, service envelope and consequence set.
Can two useful challengers be combined?
Yes, when the mechanisms are compatible, but the combination is a new candidate. Specify it before using confirmation data and test its work, interactions and rollback rather than inferring success from the separate parts.
Should the frozen reference be removed after fourth re-freeze?
It can stop controlling ordinary decisions, but keep it executable and versioned for attribution and rollback until a governed retirement decision makes that capability unnecessary.
12. Changed-context transfer, evidence check and sources
Transfer the method to a multilingual intake stream. First compare field meaning, availability and service demand by language. Build challengers around translation latency or information completeness rather than copying the source-specific winner. Hold back at least one operating cycle and verify that a capacity guard does not merely displace delay to another language queue.
NIST SP 800-137 supports organised continuous monitoring and timely risk response. The NIST engineering handbook explains cumulative-sum monitoring, including false-alarm and missed-shift design choices. These sources support the monitoring discipline; local candidate definitions, queue models and re-freeze gates still require direct validation.
Continue through the Gold Standard ecosystem
Return to the Queueing Theory subject root, the Gold Standard of Education parent, or the How X Works hub.
Immediate predecessor: third-re-freeze monitoring and fourth-thaw gates · Earlier guide: third-thaw scope design and third re-freeze gates · Queueing Theory root
These routes are grouped by learning purpose: prerequisite model, immediate predecessor, and the wider transfer framework. They are not substitutes for the evidence and limits stated in this guide.