The Gold Standard · Experimental Design
Make the decision path visible
An exception may begin as a narrow right to reproduce one historical result and gradually become a convenient path for new datasets, current decisions or routine exports. The manifest can remain technically intact while the purpose that justified it has drifted. Reauthorising that exception without challenge converts a temporary allowance into hidden policy.
The gold standard reconstructs the present use, compares it with the approved purpose, separates evidence custody from executable authority and sends material changes to an independent second line that can approve, condition, reject or retire the exception.
Choose the route closest to your decision
Chapter contents
Chapters 1–3
Chapters 4–6
Chapters 7–9
Chapters 10–12
CHAPTER 1 OF 12 · DEFINE REAUTHORISATION
1. Treat reauthorisation as a new decision
Expiry removes permission to execute or export; it does not delete retained evidence. A reauthorisation request should prove current need, scope, owner, duration, coherent environment and boundaries.
Prior approval is history, not a presumption. The present decision must stand on present facts and controls.
CHAPTER 2 OF 12 · DEFINE REAUTHORISATION
2. Write conditions that can be tested
Conditions should name permitted datasets, analytical question, users, environment, outputs, destinations, frequency, review date and prohibited uses. Connect each condition to evidence and an owner.
Words such as research only or limited use are too broad unless the system and decision process can tell whether a proposed run belongs inside them.
Evidence check
Check whether the scope, owner and decision consequence are explicit before continuing.
CHAPTER 3 OF 12 · DEFINE REAUTHORISATION
3. Separate first-line ownership from second-line approval
The first line owns the analysis, operates controls and assembles evidence. The second line is independent enough to challenge purpose, scope and risk, and has authority to refuse or condition use.
Independence is weakened when the approver benefits directly from quick access, helped design the exception or cannot inspect logs and destinations.
| Condition | Test |
|---|---|
| Dataset | Manifest match |
| Purpose | Run request fit |
| Destination | Export gate |
| Duration | Automatic expiry |
CHAPTER 4 OF 12 · DETECT PURPOSE DRIFT
4. Compare current purpose with the approved purpose
Review request language, run history, data classes, notebooks, users, outputs and downstream decisions. Mark additions, substitutions and recurring uses that were not part of the original case.
Purpose drift can occur without malicious intent. A one-off replication becomes a monthly report because the old route is convenient and already works.
CHAPTER 5 OF 12 · DETECT PURPOSE DRIFT
5. Use materiality rules for change
Require fresh approval when data sensitivity increases, new populations enter, outputs support current decisions, export destinations expand, execution becomes scheduled or the original historical question is complete.
Minor maintenance that preserves the same bounded use may be documented without restarting everything. State that threshold in advance.
Evidence check
Ask a second reviewer to reconstruct the example from the preserved evidence and stated assumptions.
CHAPTER 6 OF 12 · DETECT PURPOSE DRIFT
6. Worked example: a replication exception becomes a current comparison tool
An exception permits one team to reproduce a disputed historical subgroup result using a superseded preference-weight set. Before renewal, logs show that analysts added two current datasets and exported a monthly chart used in planning meetings. The code is reproducible, but the purpose and consequence have changed.
The owner suspends scheduled runs, preserves the historical environment and separates the original replication from the new planning request. The second line reauthorises on-demand historical reproduction only and directs the current comparison through the authorised model. Changed-context check: if a new analyst merely reruns the unchanged historical package to verify documentation, the user changes but the purpose may remain within scope after access and competence checks.
| Signal | Why material |
|---|---|
| New dataset | Population changes |
| Scheduled execution | Use becomes routine |
| Current decision | Consequence expands |
| New destination | Control boundary widens |
CHAPTER 7 OF 12 · CHALLENGE AND DECIDE
7. Assemble an evidence pack for challenge
Include the original decision, current rationale, manifest, hashes, validation result, access list, run and export logs, downstream map, unresolved limitations, incidents and proposed conditions.
The pack should let a reviewer reconstruct what can happen, not merely describe what the applicant intends.
CHAPTER 8 OF 12 · CHALLENGE AND DECIDE
8. Test the control boundary before approval
Verify blocked datasets and destinations, role permissions, scheduling, labels, effect of expired status, audit completeness and restoration from backup. Attempt plausible misuse and record the result.
Analytical reproducibility alone does not prove that the exception is safely isolated from current decision systems.
Evidence check
Verify that the control still reaches the intended decision under a changed but plausible context.
CHAPTER 9 OF 12 · CHALLENGE AND DECIDE
9. Record approve, condition, reject or retire
The decision should identify version, purpose, permitted acts, evidence, conditions, dissent, owner, expiry and immediate response to breach. Rejection should preserve required records without leaving execution enabled.
Conditional approval should say what must happen and how failure changes authority. It is not an invitation to resolve material controls after unrestricted use begins.
| Evidence | Question |
|---|---|
| Run logs | What actually happened? |
| Manifest | Is the set coherent? |
| Boundary test | What is blocked? |
| Downstream map | Who can rely on it? |
CHAPTER 10 OF 12 · CLOSE THE LOOP
10. Monitor conditions and purpose after reauthorisation
Review runs, datasets, users, exports, downstream references, incidents and organisational changes. Trigger suspension when scope expands or evidence required by a condition disappears.
Monitoring should be proportionate to the exception’s consequence. It must be frequent enough to detect drift before the next formal review.
CHAPTER 11 OF 12 · CLOSE THE LOOP
11. State the limits of second-line approval
An independent reviewer can still share assumptions, miss informal exports or rely on incomplete logs. A passed misuse test covers only the scenarios tried.
Preserve these limits and a route for users to report unexpected reuse. Approval bounds accountable authority; it does not prove misuse is impossible.
Evidence check
Confirm that versions, notices and unresolved limits remain visible in the final record.
CHAPTER 12 OF 12 · CLOSE THE LOOP
12. Audit the population of reauthorised exceptions
Track approvals, conditions, repeated renewals, purpose changes, owner changes, scheduled runs, exports, breaches, suspensions, rejected requests and retirement. Compare actual use with the authorised purpose.
The gold standard keeps historical replication available without allowing exceptions to become invisible permanent models. Every renewed authority has a present purpose, challenged boundary and accountable end point.
| Outcome | Authority |
|---|---|
| Approve | Use as stated |
| Condition | Use within obligations |
| Reject | No execution |
| Retire | Evidence only |
Frequently asked questions
Can a sound historical environment be reauthorised automatically?
No. Technical coherence does not establish current need, unchanged purpose or an acceptable operational boundary.
What makes the second line independent?
It must be able to inspect evidence, challenge the applicant and refuse or condition authority without owning the operational benefit.
Is every change purpose drift?
No. Compare the change with explicit materiality rules; maintenance may preserve purpose while new data, decisions or destinations usually expand it.
What happens after rejection?
Keep required evidence and logs, disable execution and export, and route any genuinely new use through the authorised analytical process.
Continue through verified routes
Core routes
Experimental Design subject root · Gold Standard of Education · Gold Standard directory · How X Works hub
Specialist routes
- Exception register expiry, orphaned owners and reauthorisation evidence
- Supersession compliance exceptions, historical replication and preventing exception creep
- Mixed-finding detection, decision-gate rejection and supersession compliance
- Supersession notice reach, stale analytical copies and mixed-finding prevention