The eleventh re-freeze approved a versioned physical conversion for the A90 queue-memory rule and sent missing producer metadata to an indeterminate path. Release monitoring must now protect the reference behind that conversion. A calculator can remain bit-for-bit stable while its calibration artifact, check-standard behavior or producer coverage drifts.
Reference drift is not the same as demand drift. A busy week should change queue pressure. A calibration change should alter the relationship between the observed score and the physical reference. Monitoring must preserve that distinction or it will “repair” genuine congestion by moving the ruler.
The Gold Standard is to monitor check standards, conversion residuals, metadata coverage, threshold-near actions and service consequences together; close incompatible regimes; and open a twelfth thaw only when one bounded mechanism survives changed-context replay.
A practical orientation
Use this guide after a scale-normalised A90 rule has been re-frozen. The original worked example follows three producer versions, lower/middle/upper check standards, 14,400 decisions and two independent calculators over eight weeks.
Choose a reading route
Rebuild the released reference
Chapters 1–3 freeze calibration, check standards and provenance.
Monitor reference and outcomes
Chapters 4–6 separate instrument, population and implementation drift.
Work and transfer the evidence
Chapters 7–9 calculate the signal and change producer mix and demand.
Chapter contents
- 1. Freeze the eleventh re-freeze contract
- 2. Use check standards across the calibration interval
- 3. Build a reference-provenance ledger
- 4. Separate four monitoring layers
- 5. Choose windows that do not mix reference regimes
- 6. Monitor small shifts without turning a chart into a cause
- 7. Worked example: a high-end slope drift
- 8. Transfer check: change demand and producer share
- 9. Stress uncertainty and edge decisions
- 10. Open a twelfth thaw only around the surviving mechanism
- 11. Limits, misconceptions and evidence checklist
- 12. FAQs and the next reading route
1. Freeze the eleventh re-freeze contract
The released object contains the A90 window, event-time clock, gap and interpolation rules, versioned physical conversion, calibration artifact identifier, validity interval, uncertainty, metadata requirements and indeterminate fallback. Monitoring must reconstruct this complete object rather than charting the final area alone.
Store the conversion version and input producer version with every decision. If a later calibration artifact is substituted during replay, the result answers a new question and cannot be compared with the released calculation.
- A90 definition and units.
- Calibration artifact and validity.
- Producer-version mapping.
- Uncertainty and metadata requirements.
- Indeterminate and protected fallback.
2. Use check standards across the calibration interval
A single midpoint check can miss a slope change. NIST guidance for linear calibration recommends control at the lower end, middle and upper end of the interval. Translate that idea into three stable synthetic or physical reference inputs whose expected converted values are independently known.
Run them at a declared cadence and retain raw producer output, converted value, environment, software version and uncertainty. A check standard is evidence about the measurement relationship; it is not a sample of live queue demand.
| Check point | Primary sensitivity | Failure pattern |
|---|---|---|
| Lower | Offset and low-end compression | Low queues shifted |
| Middle | Central bias | Whole regime displaced |
| Upper | Slope and saturation | Busy queues compressed or inflated |
3. Build a reference-provenance ledger
Link every calibration artifact to its source observations, fitting method, approval, uncertainty, effective versions and retirement date. A producer update may require a new artifact; a new artifact must never silently inherit the previous identifier.
The ledger also records who can promote or revoke a reference. Operational convenience is not provenance. A dashboard value with no durable source and version cannot support a thaw decision.
- Source and fit data digest.
- Method and uncertainty.
- Effective producer versions.
- Approval and revocation record.
- Decision-to-reference link.
4. Separate four monitoring layers
Layer one is check-standard stability. Layer two is production conversion residual against an independent calculator. Layer three is metadata and support coverage. Layer four is action and outcome consequence. A signal at one layer does not automatically explain another.
For example, stable check standards with rising unsupported cases suggest a metadata or routing problem, not calibration drift. Stable calculators with changing check standards suggest both implementations share a moved reference or input scale.
| Layer | Signal | First question |
|---|---|---|
| Reference | Check-standard residual | Did the scale relationship move? |
| Implementation | Independent calculator difference | Did code or numeric handling change? |
| Coverage | Unsupported or missing versions | Did provenance disappear? |
| Consequence | Action and delay changes | Did the drift matter operationally? |
5. Choose windows that do not mix reference regimes
Use per-check-period monitoring for reference stability, daily blocks for localisation and weekly windows for action consequences. Close the regime when the producer, calibration artifact, calculator, clock basis or gap rule changes.
A control chart assumes a stable reference process. Re-estimating the centre from a window that already contains drift can make the new error look normal. Keep the released reference visible and start a new regime only through an explicit decision.
| Window | Use | Do not infer alone |
|---|---|---|
| Check period | Reference health | Population consequence |
| Daily block | Localise version or route | Persistent policy drift |
| Week | Action and outcome pattern | Causal mechanism |
| Release regime | Comparable history | Future transfer |
6. Monitor small shifts without turning a chart into a cause
Shewhart limits are intuitive for large changes; an exponentially weighted moving average can be more sensitive to gradual drift because it retains diminishing weight from prior measurements. Keep the weighting parameter, centre and limits versioned.
A chart signal opens reconstruction. It does not prove heat, wear, software or calibration damage. Record evidence for competing mechanisms and repeat the check before rejecting a calibration run when the safety policy permits.
| Statistic | Strength | Limit |
|---|---|---|
| Current residual | Fast large-change view | Weak for slow drift |
| EWMA | Sensitive to gradual movement | Depends on historical reference |
| Coverage rate | Finds missing support | Not measurement accuracy |
| Action reversal rate | Shows consequence | Confounded by demand mix |
7. Worked example: a high-end slope drift
For six weeks, lower, middle and upper checks stay within their released envelopes. In week seven the upper check rises by 3.8%, the middle by 0.7% and the lower remains stable. Both calculators agree, metadata coverage is 99.9%, and live high-pressure queues show larger converted A90 values.
Among 2,400 week-seven decisions, 126 cross the transfer threshold earlier. Replaying the same raw tapes with the last known-good calibration restores 119 actions. Seven remain different because their queue observations genuinely changed. The pattern localises the main defect to the high-end calibration slope rather than the A90 implementation.
A replacement artifact fitted with the three check standards restores all checks within uncertainty. Shadow replay keeps protected fallback within its gate and introduces no new unsupported cases.
| Evidence | Released artifact | Replacement artifact |
|---|---|---|
| Lower check | +0.1% | 0.0% |
| Middle check | +0.7% | +0.1% |
| Upper check | +3.8% | +0.2% |
| Early transfers versus reference | 126 | 7 |
| Unsupported cases | 2 | 2 |
8. Transfer check: change demand and producer share
First replay a high-arrival event. The replacement calibration should preserve the genuine rise in physical queue pressure rather than recenter it. Then increase the affected producer from 15% to 65% while keeping case tapes fixed; each case result should remain unchanged.
Finally replay the oldest still-supported producer version and an intentionally unsupported future version. The first should use its declared artifact; the second should become indeterminate rather than borrowing the latest conversion.
| Changed context | Expected invariant | Failure exposed |
|---|---|---|
| High demand | Real pressure remains visible | Population recentering |
| Producer share | Case result unchanged | Mixture dependence |
| Older supported version | Mapped artifact used | Latest-only lookup |
| Future version | Indeterminate | Unsafe default |
9. Stress uncertainty and edge decisions
Propagate calibration uncertainty through A90 and mark actions that can reverse within the supported interval. Do not present a point estimate as certainty when the threshold lies inside that band.
Build a boundary-heavy challenge set in addition to natural traffic. The natural set estimates exposure; the challenge set reveals sensitivity. Keep them separate so an intentionally difficult sample is not mistaken for prevalence.
- Uncertainty propagated to A90.
- Natural and boundary-heavy samples.
- Numeric and action tolerances separate.
- Protected outcomes retained.
- No tuning on final challenge set.
10. Open a twelfth thaw only around the surviving mechanism
The twelfth-thaw gate requires persistent reference evidence, exact raw-tape reconstruction, a named layer, action or protected consequence, and a replay question changing one mechanism. A broad proposal to modernise normalisation does not pass.
In the example, replacing a damaged high-end calibration artifact is ordinary reference control if it restores the already approved relationship. A twelfth thaw is justified only if the supported conversion model itself must change—for example, from linear to piecewise—because the old model no longer meets uncertainty and transfer gates.
| Gate | Ordinary repair | Twelfth thaw |
|---|---|---|
| Contract | Restores released relationship | Changes supported model |
| Evidence | Known-good reference replay | Competing model comparison |
| Scope | Artifact replacement | Bounded conversion redesign |
| Decision | Reissue reference | Re-freeze policy contract |
11. Limits, misconceptions and evidence checklist
Stable calibration does not prove that the queue classifier measures every relevant service condition, and stable A90 does not prove the allocation policy is fair. Check standards protect the measurement relationship inside their interval; extrapolation remains unsupported.
Preserve check observations, reference artifacts, uncertainty, raw tapes, calculator builds, coverage, chart parameters, regime boundaries, action outcomes, exclusions, reviewer decisions and rollback instructions. Reproduce the evidence on a clean calculator.
- No extrapolation beyond the interval.
- Stable chart is not causal proof.
- Reference and demand drift separated.
- Shared upstream errors tested.
- Clean reproduction required.
12. FAQs and the next reading route
Release monitoring protects the ruler behind a normalised decision, not only the number printed by the calculator. That makes later repairs smaller and keeps genuine queue change visible.
Read the Queueing Theory root, the tenth re-freeze monitoring article and the eleventh-thaw comparison before this guide. The Gold Standard parent and How X Works hub connect the same practice across subjects: define the reference, preserve common evidence, test transfer and state limits.
Why are three check standards useful?
They can distinguish offset, central and slope changes across the calibration interval; one midpoint may miss a high-end defect.
Does an EWMA identify the cause of drift?
No. It can make gradual change easier to detect, but reconstruction and independent evidence are needed to identify a mechanism.
When does a drift repair require a thaw?
When restoring the released artifact is insufficient and the supported conversion model or decision contract must change.
Continue through the verified ecosystem
Sources and scope note
- NIST/SEMATECH — Instrument control for linear calibration — used for lower, middle and upper check standards and ongoing calibration control.
- NIST/SEMATECH — Drift — used for distinguishing slow response drift and documenting its maximum change and uncertainty.
- NIST/SEMATECH — EWMA control chart — used for gradual-drift monitoring and dependence on representative historical data.
The queue, calibration artifacts, thresholds and outcomes are original teaching examples. The NIST sources support measurement-control principles; they do not validate the example policy or gates.