The Gold Standard · Ecology
Make the decision path visible
Recalibration is a point-in-time achievement. New land uses, sensor changes, remedy ageing and reviewer workarounds can make yesterday’s challenge set unrepresentative. Sentinel cases are meant to reveal that decay before a real protectiveness signal is missed.
The gold standard treats sentinel failure as evidence about the model’s boundary, not as a score to hide. It contains affected decisions, diagnoses the change and chooses repair, restricted use or retirement according to current protectiveness needs.
Choose the route closest to your decision
Chapter contents
Chapters 1–3
Chapters 4–6
Chapters 7–9
Chapters 10–12
CHAPTER 1 OF 12 · DEFINE DECAY
1. Treat validation as time-bounded
Record the model version, data period, site assumptions, sentinel set, performance boundary and review date. State which changes invalidate the claim early.
Validation supports use under described conditions; it is not a permanent certificate.
CHAPTER 2 OF 12 · DEFINE DECAY
2. Design sentinel cases around consequences
Include land-use changes, remedy failures, new receptors, benign duplicates, missing feeds and rare combinations. Give each an expected tier and action.
A sentinel should test whether the right decision happens, not merely whether a score stays within tolerance.
Evidence check
Check whether the scope, owner and decision consequence are explicit before continuing.
CHAPTER 3 OF 12 · DEFINE DECAY
3. Define failure and escalation rules
Distinguish one corrupted test record from repeated misclassification, pathway-specific deterioration and system-wide decay. Link each level to investigation and containment.
Predetermined rules stop teams from dismissing inconvenient failures or retiring a useful model after one bad fixture.
| Field | Evidence |
|---|---|
| Version | Model used |
| Data period | Known context |
| Sentinel set | Consequential cases |
| Review date | Freshness point |
CHAPTER 4 OF 12 · RESPOND TO FAILURE
4. Preserve the failed run
Capture inputs, feature transformations, model version, configuration, result, expected result and environment. Check whether the sentinel itself changed.
Without this record, reviewers cannot tell model decay from a broken test harness.
CHAPTER 5 OF 12 · RESPOND TO FAILURE
5. Contain the affected pathway
Restore manual or rule-based review for conditions represented by the failed sentinel. Pause only approval decisions that depend on the uncertain tier.
Containment should protect people and environment while keeping unaffected, independently controlled work moving.
Evidence check
Ask a second reviewer to reconstruct the example from the preserved evidence and stated assumptions.
CHAPTER 6 OF 12 · RESPOND TO FAILURE
6. Worked example: a feed recode defeats a receptor sentinel
A sentinel for sensitive land use is expected to enter high-priority review. After a source-system update, the childcare code changes format and the feature mapper treats it as unknown-low. The sentinel fails while routine model metrics remain stable.
The owner routes all new land-use records to manual review, repairs the mapping and searches live decisions since the recode. Changed-context check: if the sentinel file alone used an obsolete code while production data and other challenge cases remain correct, repair the test asset and document why model validity was not lost.
| Level | Response |
|---|---|
| Test asset error | Repair fixture |
| Local decay | Restrict pathway |
| Broad decay | Suspend model |
| Repeated patch failure | Retire |
CHAPTER 7 OF 12 · REPAIR OR RETIRE
7. Measure the decay boundary
Test related codes, locations, pathways and time periods. Compare live outcomes, manual rescues and other sentinels to determine whether failure is local or systemic.
The breadth of evidence determines whether a patch, partial restriction or full retirement is proportionate.
CHAPTER 8 OF 12 · REPAIR OR RETIRE
8. Compare repair with replacement and retirement
Estimate whether the model can be revalidated with current data and interpretable controls. Consider a simpler rule set or human review when patches accumulate faster than evidence.
Retirement can be the safer engineering choice; it is not an admission that every past decision was wrong.
Evidence check
Verify that the control still reaches the intended decision under a changed but plausible context.
CHAPTER 9 OF 12 · REPAIR OR RETIRE
9. Move to a successor without losing protection
Define the interim route, transfer alerts and history, validate successor decisions and keep rollback evidence. Mark the old model non-authoritative in every consumer.
A replacement is incomplete if dashboards or exports continue to rely on the retired score.
| Option | Use when |
|---|---|
| Patch | Cause narrow |
| Recalibrate | Relationship changed |
| Replace | Better control available |
| Retire to manual | Model no longer defensible |
CHAPTER 10 OF 12 · PROTECT CONTINUITY
10. Reconcile the decay window
Trace affected approvals and alerts from the earliest plausible change. Reassess material cases under the repaired or successor process.
Preserve stable outcomes and document uncertainty where evidence cannot be recreated.
CHAPTER 11 OF 12 · PROTECT CONTINUITY
11. State retirement limits
A successor may share data dependencies and blind spots. Manual review can fatigue, and sentinel sets cannot represent every ecological change.
Keep raw observations, independent inspections and renewal triggers outside the model.
Evidence check
Confirm that versions, notices and unresolved limits remain visible in the final record.
CHAPTER 12 OF 12 · PROTECT CONTINUITY
12. Audit model validity through retirement
Track sentinel failures, time to containment, live misses, repair attempts, scope restrictions, successor validation and residual consumers.
The gold standard retires a model before familiarity becomes false confidence, while preserving the evidence needed to understand earlier decisions.
| Measure | Warning |
|---|---|
| Old score consumed | Authority persists |
| Shared dependency | Successor vulnerable |
| Unreviewed window | Exposure unknown |
| No trigger owner | Continuity weak |
Frequently asked questions
Does one sentinel failure require model retirement?
No. Verify the test and determine the failure boundary, while containing any affected high-consequence pathway.
Why not keep patching a familiar model?
Accumulated patches can make scope and evidence opaque; replacement or retirement may offer clearer protection.
What happens to earlier approvals?
Review those in the plausible decay window where the failed pathway could be material; do not invalidate unrelated outcomes automatically.
When is retirement complete?
When the successor route is validated, the old model is non-authoritative everywhere and affected decisions have been reconciled.
Continue through verified routes
Core routes
Ecology subject root · Gold Standard of Education · Gold Standard directory · How X Works hub
Specialist routes
- Alert-triage model drift, missed protectiveness signals and surveillance recalibration
- Enhanced surveillance renewal fatigue, alert burden and protectiveness-focused exit
- Monitoring retirement reversal, renewal triggers and returning to enhanced surveillance
- Second-closure surveillance, new residual signals and final monitoring retirement