A fifth re-freeze is a controlled operating state, not the end of learning. Once the Q5 rule is live, the team must watch whether its mechanism, eligible population, reset-portfolio lineage and service assumptions still hold. A movement in one headline rate is not enough to identify which of those layers changed.
This guide continues the fifth-thaw portfolio-reset work. It shows how to monitor the released rule beside its Q0 rollback reference and component challengers, measure reset-portfolio coverage, separate candidate drift from arrival composition and capacity stress, and open a sixth thaw only for the unsupported boundary.
The Gold Standard is a diagnosis that preserves alternatives. Monitoring should reveal whether Q5 itself changed, whether the world moved beyond the reset portfolio, or whether the service system changed around an otherwise stable rule. Each explanation leads to a different response.
A practical orientation
Keep six clocks visible: decision-time information, assignment, service start, completion, outcome availability and monitoring review. A late field or delayed outcome can make a stable rule look different if the comparison window silently changes.
The worked example follows a mobile operating-system update that changes field delivery and burst timing. The aggregate changed-assignment rate rises. Stratification shows that Q5 is stable where its original candidates apply, while a new incomplete-field state falls outside the reset portfolio and creates the true coverage gap.
- Freeze the monitoring contract — for owners preserving the fifth re-freeze baseline.
- Measure portfolio drift — for analysts separating coverage, ranking and implementation.
- Follow the mobile update — for a worked diagnosis with common clocks.
- Apply sixth-thaw gates — for bounded governance and rollback.
- Transfer the method — for another population or demand shock.
Chapter contents
Open a route that matches the decision in front of you, or read straight through. Each route returns to the same evidence chain.
Start with the operating question
Build the evidence chain
Define reset-portfolio drift precisely
Work the example
Worked example: a mobile update changes coverage, not Q5 logic
1. Freeze the fifth-re-freeze monitoring contract
Record the live Q5 implementation, eligible population, feature definitions, thresholds, reason codes, service-capacity envelope and rollback reference. Store the Q1 and Q4 component outputs when feasible, because the combined rule can remain stable while one mechanism loses coverage.
Define review cadence, minimum denominators, expected reporting delay and the authority for pause, rollback or sixth thaw. Monitoring that changes its reference after seeing the data cannot distinguish a new signal from a new ruler.
- Executable Q5 and Q0 versions with hashes or release identifiers.
- Decision-time field and population definitions.
- Reset-portfolio membership and candidate eligibility reasons.
- Ordinary and stress capacity envelopes.
- Protected outcomes, action limits and named decision owners.
2. Build a layered dashboard instead of one success rate
Separate input, decision, changed-set, service and outcome layers. Input measures describe who arrived and what information was present. Decision measures show assignments and reason codes. Service measures show donated and received work, waiting tails and completion. Outcomes test whether the intended protection transferred.
Report denominators beside rates. A fall from 8% to 5% means something different when the eligible population shrinks, outcome maturity changes or the missingness state is reclassified. Include stable Q0 comparisons on the same records rather than comparing Q5 today with Q0 months ago.
| Layer | Example measure | Question answered |
|---|---|---|
| Inputs | Arrival mix and decision-time completeness | Did the population or information state change? |
| Decisions | Q5 assignment and changed-set rate | Did the rule act differently? |
| Service | Specialist minutes and upper-tail wait | Did operational consequences change? |
| Outcomes | Protected result after maturity | Did the intended benefit transfer? |
3. Define reset-portfolio drift precisely
Coverage drift occurs when more arrivals fall outside the states represented by Q0, Q1, Q2, Q4 and Q5. Ranking drift occurs when the relative performance of eligible candidates changes. Implementation drift occurs when the live code no longer matches the frozen definition. These are not interchangeable.
A reset portfolio may also become stale because a candidate’s eligibility premise changes. If a field once available at decision time now arrives later, the candidate has not merely scored worse; part of its valid operating domain has changed. Preserve that distinction in the alert.
| Drift type | Diagnostic comparison | First response |
|---|---|---|
| Coverage | Share outside declared states | Characterise the new state |
| Ranking | Common replay of eligible candidates | Check mechanism and uncertainty |
| Implementation | Live versus frozen decisions | Repair or roll back |
| Capacity | Same decisions under changed service load | Protect service before rule redesign |
4. Use common clocks and stratification before declaring drift
Align arrivals by decision time and evaluate each rule with the information then available. Stratify by channel, device family, information state, burst condition and service regime declared in the monitoring plan. A useful stratum has enough support to interpret; a tiny subgroup should be reported as uncertain rather than forced into a verdict.
Compare like with like across outcome maturity. If the newest arrivals have not completed their follow-up window, keep them in decision and service monitoring but exclude them from mature outcome denominators. Otherwise a release can look safer or worse simply because its outcomes are unfinished.
5. Detect sustained changes without confusing signal and cause
A cumulative-sum chart can detect a small persistent movement relative to a fixed reference more quickly than a chart designed only for large jumps. Set its reference, target shift and action rule before use, and accompany the signal with the raw rate, denominator and stratum evidence.
A signal answers whether the monitored series moved under its statistical design. It does not tell the team whether the cause is Q5, arrival composition, missing fields, staffing or a logging change. Diagnosis still requires version, input and service evidence.
6. Worked example: a mobile update changes coverage, not Q5 logic
After a mobile operating-system update, the aggregate Q5 changed-assignment rate rises from 11% to 16% and specialist waiting increases. The team first verifies that the Q5 hash and reason-code logic are unchanged. It then reconstructs decision-time field delivery and discovers a new state: one field arrives after the decision for 18% of updated devices but was present for nearly all devices in the fifth-thaw replay.
Within the original complete-field and declared incomplete-field strata, Q5 versus Q0 performance remains inside the monitoring envelope. The new late-field state is not represented by Q1’s original missingness definition, so the reset portfolio covers only 82% of current updated-device arrivals. Capacity stress is real, but it is a consequence of the uncovered state rather than proof that Q5 reversed everywhere.
| Check | Observed evidence | Interpretation |
|---|---|---|
| Implementation | Q5 version and common-record decisions match | No code drift |
| Original strata | Assignment and outcome differences remain stable | Supported scope still holds |
| New late-field state | 18% of updated-device arrivals | Portfolio-coverage gap |
| Service | Specialist tail rises in that state | Immediate capacity protection needed |
| Conclusion | Bounded unsupported boundary | Candidate for sixth thaw |
7. Protect service while preserving diagnostic evidence
Apply the existing capacity guard to the uncovered state and retain Q5 for supported strata. If the tail threatens a protected limit, route the affected state to Q0 or a declared safe fallback while continuing shadow evaluation. Record the control as an operational containment, not as evidence that a new candidate has succeeded.
Preserve arrival snapshots, alternative candidate outputs and donated and received work. Broadly replacing Q5 would erase evidence from the stable population and expand risk beyond the diagnosed boundary.
8. Set sixth-thaw admission gates
Open a sixth thaw only when the unsupported state persists, the monitoring contract remains valid, implementation and logging explanations have been checked, operational containment is defined and a mechanism-linked candidate can be evaluated. The thaw scope should name the affected device and information state, not the whole mobile channel.
A gate can admit investigation without admitting deployment. Separate authority to explore candidates, run common replays, begin limited release and re-freeze. Each stage needs its own evidence and rollback condition.
- Persistent signal with declared denominators and mature windows.
- Frozen Q5 implementation verified or implementation drift repaired first.
- Population, information and capacity alternatives assessed.
- Unsupported boundary and protected scope stated.
- Safe containment and rollback executable.
- Fresh confirmation evidence and authority assigned.
9. Choose among monitor, repair, roll back and thaw
Continue monitoring when evidence remains compatible with ordinary variation and no protected limit is threatened. Repair when the live implementation or logging differs from the frozen contract. Roll back when immediate harm cannot be contained narrowly. Thaw when the operating world has moved beyond an otherwise correctly implemented rule.
These decisions can coexist by scope. The team may keep Q5 in supported strata, roll the new state to Q0 and open a sixth thaw for that state. A single system-wide label hides this useful control.
10. State the limits of surveillance and reset-portfolio evidence
Historical replays cannot reconstruct behaviour changes caused by the live rule, and component outputs may share errors because they use the same records. Sparse states, delayed outcomes and changing service practice widen uncertainty. State those limits before using a ranking change to redesign the rule.
The portfolio contains imagined alternatives, not every possible mechanism. Stable performance among declared candidates does not prove optimality, while poor coverage does not prove that any available candidate will work. The defensible conclusion is conditional on the tested scope and evidence.
11. Misconceptions, practical questions and transfer
Common mistakes include watching only aggregate accuracy, treating capacity stress as model drift, changing CUSUM limits after an alert, and interpreting a new population state as failure of every existing candidate. The repair is a frozen contract, layered evidence and scope-specific action.
Does a sixth-thaw alert require immediate replacement of Q5?
No. It requires diagnosis and protection. Keep Q5 where its support remains valid, contain the unsupported state and admit only the bounded question to the thaw.
Can the reset portfolio drift even when Q5 code is unchanged?
Yes. Coverage, candidate eligibility, input delivery and service assumptions can change around stable code. That is why portfolio and operating-state evidence sit beside implementation checks.
12. Changed-context evidence check and authoritative sources
Transfer the method to a partner campaign that doubles arrivals without changing field delivery. Here the likely first explanation is capacity, not reset-portfolio coverage. Keep the same frozen rule and common clocks, but stress the service envelope before admitting a sixth thaw. A genuine transfer changes the diagnosis when the mechanism changes.
NIST describes organisation-wide continuous monitoring and timely risk response in SP 800-137. NIST’s engineering statistics handbook explains cumulative-sum control charts, including their use for detecting sustained small shifts. These sources support the monitoring discipline; the queue definition, causal alternatives and operational gates remain local responsibilities.
Continue through the Gold Standard ecosystem
Return to the Queueing Theory subject root, the Gold Standard of Education parent, or the How X Works hub.
Immediate predecessor: fifth-thaw scope, portfolio-reset replays and fifth re-freeze · Earlier guide: fourth-re-freeze monitoring and fifth-thaw gates · Queueing Theory root
These routes are grouped by learning purpose: prerequisite model, immediate predecessor, and the wider transfer framework. They are not substitutes for the evidence and limits stated in this guide.