The Gold Standard · Archaeology
Make the decision path visible
After root-cause review, a control redesign may reduce visible override rates yet increase hidden exclusions, delay releases or weaken archaeological traceability. Teams can all be looking at correct numbers while answering different questions. The dispute is not solved by choosing the most reassuring metric.
The gold standard defines success before validation, connects each criterion to the protected decision, tests the redesign against a frozen baseline and changed contexts, and gives governance a clear choice: accept, condition, reject or require further evidence.
Choose the route closest to your decision
Chapter contents
Chapters 1–3
Chapters 4–6
Chapters 7–9
Chapters 10–12
CHAPTER 1 OF 12 · FRAME SUCCESS
1. Start with the decision the redesign must protect
Name the bridge-model release decision, the failure being prevented and the archaeological evidence that must remain traceable. A control is successful only in relation to that protected consequence.
This prevents a convenient operational metric from replacing the real purpose. Lower queue time is useful, but not if it is achieved by excluding difficult stratigraphic associations from review.
CHAPTER 2 OF 12 · FRAME SUCCESS
2. Separate necessary criteria from desirable outcomes
Necessary criteria are pass-fail protections such as no unreviewed override, preserved provenance and complete release-gate evidence. Desirable outcomes may include faster review, fewer false alarms or lower reviewer burden.
Write the hierarchy before results are known. Otherwise each group can promote the measure that favours its preferred design.
Evidence check
Check whether the scope, owner and decision consequence are explicit before continuing.
CHAPTER 3 OF 12 · FRAME SUCCESS
3. Define observable measures and decision rules
For every criterion, specify population, numerator, denominator, time window, uncertainty, threshold and owner. State how missing or ambiguous cases are handled.
A metric without a denominator can improve because hard cases disappeared. A threshold without uncertainty can turn ordinary variation into a false dispute.
| Level | Example |
|---|---|
| Protected claim | No unreviewed release |
| Necessary criterion | Provenance complete |
| Performance criterion | Review delay |
| Context measure | Case complexity |
CHAPTER 4 OF 12 · VALIDATE FAIRLY
4. Freeze the validation protocol and baseline
Record the old control, redesign version, test cases, expected outcomes, historical comparison period, exclusions and change log. Preserve the data and code needed to reproduce the assessment.
Protocol changes may be justified when a flaw is found, but they should be versioned and reviewed. Quietly changing success criteria after seeing results destroys the comparison.
CHAPTER 5 OF 12 · VALIDATE FAIRLY
5. Use independent challenge and counterexamples
Ask a reviewer outside the redesign team to test boundary cases, dependency failures, rare site contexts and incentives to bypass the gate. Seek cases that could satisfy the headline metric while violating the protected claim.
Independence does not require ignorance of operations. It requires freedom to report an adverse result and a clear route to governance when interpretations diverge.
Evidence check
Ask a second reviewer to reconstruct the example from the preserved evidence and stated assumptions.
CHAPTER 6 OF 12 · VALIDATE FAIRLY
6. Worked example: override rate falls while coverage loss rises
A redesigned bridge-model gate cuts recorded overrides from 12 percent to 3 percent. Operations call it successful. Independent review finds that uncertain cross-layer associations are now routed outside the measured queue, so review coverage falls from 96 percent to 78 percent and provenance defects increase.
The panel rejects override rate as sufficient evidence, restores the missing population to the denominator and conditions acceptance on coverage and provenance thresholds. Changed-context check: if coverage stays at 96 percent and the remaining override decline persists across a high-complexity excavation season, the same lower rate becomes credible evidence of improvement.
| Type | Resolution path |
|---|---|
| Definition | Agree population |
| Measurement | Validate method |
| Threshold | Governance choice |
| Causation | Add comparison |
CHAPTER 7 OF 12 · RESOLVE THE DISPUTE
7. Map each disagreement to consequence and evidence
Classify disputes as definition, measurement, threshold, causal attribution or value trade-off. Ask what evidence would change each position and what protected outcome is at stake.
This keeps a disagreement about acceptable delay from being disguised as a technical argument about data quality. Some conflicts need more testing; others need an explicit governance choice.
CHAPTER 8 OF 12 · RESOLVE THE DISPUTE
8. Compare designs across a balanced evidence set
Use ordinary cases, known failures, rare boundary cases, changed workloads and degraded dependencies. Report false acceptance, false rejection, coverage, traceability, delay and reviewer burden together.
No single score preserves every value. A balanced set makes trade-offs visible and prevents local optimisation from masquerading as overall control effectiveness.
Evidence check
Verify that the control still reaches the intended decision under a changed but plausible context.
CHAPTER 9 OF 12 · RESOLVE THE DISPUTE
9. Make governance acceptance a bounded decision
The decision record should identify the accepted version, criteria met, unresolved evidence, conditions, monitoring period, owner, re-escalation triggers and review date. Record dissent and why it did or did not change the outcome.
Conditional acceptance is not vague compromise. It is permission within stated limits, backed by obligations that can be tested and revoked.
| Measure | Failure revealed |
|---|---|
| Coverage | Hidden exclusion |
| False acceptance | Weak protection |
| False rejection | Unnecessary delay |
| Traceability | Broken evidence chain |
CHAPTER 10 OF 12 · KEEP ACCEPTANCE HONEST
10. Monitor the conditions after deployment
Track whether the validation population remains representative, whether new bypass routes appear and whether thresholds behave under real load. Compare operational evidence with the accepted version and protocol.
Post-acceptance monitoring is part of validation, not an excuse to deploy an unsafe control. Necessary protections should pass before release; monitoring addresses residual uncertainty.
CHAPTER 11 OF 12 · KEEP ACCEPTANCE HONEST
11. State the limits of validation and acceptance
Historical cases may not represent future excavation contexts, rare dependencies or strategic behaviour. Independent reviewers can share blind spots, and stable averages can hide subgroup harm.
State these limits beside the decision. Acceptance means the evidence is sufficient for a bounded use, not that the redesign is universally correct.
Evidence check
Confirm that versions, notices and unresolved limits remain visible in the final record.
CHAPTER 12 OF 12 · KEEP ACCEPTANCE HONEST
12. Audit criteria, disputes and later performance
Track protocol versions, excluded cases, changed denominators, unresolved dissent, conditions, re-escalations, bypasses and post-acceptance failures. Compare claims at approval with evidence in operation.
The gold standard makes a redesign defensible because success criteria stay connected to archaeological traceability, competing evidence remains visible and governance owns the final trade-off.
| Field | Purpose |
|---|---|
| Version | Bound the decision |
| Conditions | Required obligations |
| Trigger | Return to review |
| Dissent | Preserve challenge |
Frequently asked questions
Can a redesign pass one criterion and still be rejected?
Yes. Failure of a necessary protection can outweigh improvement in a desirable performance measure.
Who should set the success criteria?
Control owners, affected users, independent reviewers and governance should agree the hierarchy before results are known.
Is conditional acceptance weaker than full acceptance?
It is more precise: use is allowed only within recorded limits and while specified obligations remain satisfied.
What happens when reviewers still disagree?
Record the dispute, evidence and consequences, then route the unresolved trade-off to the authority accountable for the protected decision.
Continue through verified routes
Core routes
Archaeology subject root · Gold Standard of Education · Gold Standard directory · How X Works hub
Specialist routes
- Regression root-cause disputes, independent review and control redesign
- Remediation regression, control-drift signals and re-escalation
- Override remediation effectiveness, repeat-use triggers and governance escalation
- Override recurrence, exception normalisation and restoring release-gate discipline