THE GOLD STANDARD OF JUDGEMENT · SPECIALIST STUDY
Use nonstandard aggregation only after naming the property you relaxed
Pre-aggregation functions and penalty-based or directional means can retain monotonic behaviour along selected directions while relaxing ordinary boundary or full-coordinate monotonicity. The gold standard states the direction, generator, penalty, domain and failed axioms before using the result in judgement.
This guide belongs to the Judgement subject guide, the Gold Standard of Education, and the How X Works learning library. Choose a reading route, work the examples, then use the transfer checks to test whether the method survives a changed context.
Choose a reading route
Build the foundations · chapters 1–2
Apply the core method · chapters 3–5
Test evidence and alternatives · chapters 6–7
Handle limits and disagreement · chapters 8–9
Transfer and govern · chapters 10
The whole problem at a glance
| Layer | Question | Control |
|---|---|---|
| Scope | What is included? | Published boundary |
| Evidence | What supports the step? | Traceable record |
| Decision | What follows? | Named authority |
| Revision | What can change it? | Review trigger |
A label or score is not a substitute for its evidence path.
CHAPTER 1 OF 10 · Build the foundations
1. Start from the ordinary aggregation axioms
Core idea. Build the foundations
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Boundary, monotonicity and idempotency give familiar meaning.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. A weighted mean is audited before any relaxation. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 1 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 2 OF 10 · Build the foundations
2. Define directional monotonicity
Core idea. Build the foundations
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. A function may increase only along a chosen vector direction.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. Both inputs move together along a positive ray and output cannot fall. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 2 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 3 OF 10 · Apply the core method
3. Distinguish pre-aggregation from aggregation
Core idea. Apply the core method
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Relaxed monotonicity creates a broader operator class.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. A pre-aggregation function is labelled with its exact retained properties. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 3 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 4 OF 10 · Apply the core method
4. Construct penalty-based means
Core idea. Apply the core method
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. A penalty function selects the representative value minimising disagreement.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. Absolute deviation yields a median-like result; squared loss yields a mean-like result. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 4 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 5 OF 10 · Apply the core method
5. Use directional means transparently
Core idea. Apply the core method
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Directional movement can model coupled improvement or evidence change.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. A two-criterion score rises when both criteria improve together. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 5 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
Worked evidence path
| Stage | Ordinary case | Boundary case |
|---|---|---|
| Input | Complete and verified | Missing or conflicting |
| Method | Declared rule | Assumption-sensitive |
| Output | Supported conclusion | Unresolved or bounded |
| Review | Routine check | Independent escalation |
Do not force an answer when the method says unresolved.
CHAPTER 6 OF 10 · Test evidence and alternatives
6. Test boundaries and idempotency
Core idea. Test evidence and alternatives
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Nonstandard boundaries can produce surprising extreme results.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. All-zero, all-one and equal-input vectors are tested explicitly. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 6 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 7 OF 10 · Test evidence and alternatives
7. Prove or exhaustively test retained properties
Core idea. Test evidence and alternatives
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. A named operator needs evidence for every claimed property.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. Finite tables enumerate comparable pairs and direction steps. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 7 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 8 OF 10 · Handle limits and disagreement
8. Compare simpler rules and standard means
Core idea. Handle limits and disagreement
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Complex operators should add justified meaning.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. The same cases are run through median, weighted mean and penalty operator. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 8 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
Governance checks
| Risk | Signal | Response |
|---|---|---|
| Drift | Definitions change | Version and reconcile |
| Bias | Groups or pathways differ | Disaggregate and review |
| Gaming | Metric becomes target | Countermetric and file audit |
| Error | New evidence arrives | Correct downstream use |
Correction must reach future decisions.
CHAPTER 9 OF 10 · Handle limits and disagreement
9. Communicate failed axioms and sensitivity
Core idea. Handle limits and disagreement
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Readers need to know where ordinary intuition stops.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. Documentation lists a counterexample to full monotonicity. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 9 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
CHAPTER 10 OF 10 · Transfer and govern
10. Govern parameter and direction choice
Core idea. Transfer and govern
Method. Define the input, scope, governing assumptions and intended output before calculating. Preserve original evidence, write each transformation or judgement step, and keep an explicit unresolved state. The method should let another authorised reviewer reproduce the result without guessing which rule or denominator was used.
Worked example. Directions and penalties are empirical and normative choices.
Changed-context transfer. Transfer the method to a school or public-service workflow. Identify which formal structure remains valid, then replace the domain assumptions, authority, privacy controls and remedy route rather than copying the conclusion.
Limits. A review records who selected them and which outcomes reverse. Treat a timeout, missing record, undefined cell or protected fact as its own state; do not convert it into zero, false or consent.
Evidence and review. For this chapter, keep a dated evidence record: the input, rule or assumption, result, accountable owner and a condition that would change the conclusion. Work an ordinary case and a boundary case, preserve rejected alternatives, and require an authorised independent reader to reproduce where judgement entered. Chapter 10 is complete only when the claimed result survives that review.
Misconception to avoid. A polished number or table makes the result objective. In Judgement, the result remains conditional on definitions, coverage, properties, evidence and authority.
Practical evidence check
- Work one ordinary case and one boundary case, change one material assumption, and record exactly when the result reverses or becomes unresolved.
- Return to chapter contents
Build a specialist evidence record
Record scope, definitions, source evidence, version and accountable owner.
Work one ordinary and one boundary case, including rejected alternatives.
Run sensitivity against the assumption most likely to reverse the result.
Assign correction, retention, review and retirement rules.
Build a decision record
Choose one consequential workflow and state the decision, the people affected, the time boundary and the claim being tested. Record original evidence before interpretation so later review can distinguish source material from inference.
Apply the article’s method to one routine case and one case near a boundary. Show the calculation or rule path, identify missing information and explain why plausible alternatives were accepted or rejected.
Change one material assumption at a time. Record what remains stable, what reverses and what becomes unresolved. Missing evidence must remain visible rather than being converted into a convenient zero, default or certainty.
Compare the result with at least one simpler alternative: explicit-only reading, an unweighted description, a transparent rule or a basic case-flow count, as appropriate. If the complex method does not improve the decision or explanation, prefer the simpler method. Record both outputs so later reviewers can see what the added structure contributed and what it obscured.
Invite a reviewer who was not involved in design to challenge scope, definitions, denominators, boundary conditions and incentives. Ask what evidence is systematically absent, who bears the cost of error, and how the method could be gamed. Convert each credible challenge into a test, collection action or explicit unresolved limitation rather than a defensive footnote.
Assign an owner, correction route, retention rule and review date. Verify that correction reaches derived outputs and future decisions, then retire the method when its assumptions, authority or evidence base no longer hold.
Run a boundary-case audit
Choose one consequential workflow and state the decision, the people affected, the time boundary and the claim being tested. Record original evidence before interpretation so later review can distinguish source material from inference.
Apply the article’s method to one routine case and one case near a boundary. Show the calculation or rule path, identify missing information and explain why plausible alternatives were accepted or rejected.
Change one material assumption at a time. Record what remains stable, what reverses and what becomes unresolved. Missing evidence must remain visible rather than being converted into a convenient zero, default or certainty.
Compare the result with at least one simpler alternative: explicit-only reading, an unweighted description, a transparent rule or a basic case-flow count, as appropriate. If the complex method does not improve the decision or explanation, prefer the simpler method. Record both outputs so later reviewers can see what the added structure contributed and what it obscured.
Invite a reviewer who was not involved in design to challenge scope, definitions, denominators, boundary conditions and incentives. Ask what evidence is systematically absent, who bears the cost of error, and how the method could be gamed. Convert each credible challenge into a test, collection action or explicit unresolved limitation rather than a defensive footnote.
Assign an owner, correction route, retention rule and review date. Verify that correction reaches derived outputs and future decisions, then retire the method when its assumptions, authority or evidence base no longer hold.
Test independent reproduction
Choose one consequential workflow and state the decision, the people affected, the time boundary and the claim being tested. Record original evidence before interpretation so later review can distinguish source material from inference.
Apply the article’s method to one routine case and one case near a boundary. Show the calculation or rule path, identify missing information and explain why plausible alternatives were accepted or rejected.
Change one material assumption at a time. Record what remains stable, what reverses and what becomes unresolved. Missing evidence must remain visible rather than being converted into a convenient zero, default or certainty.
Compare the result with at least one simpler alternative: explicit-only reading, an unweighted description, a transparent rule or a basic case-flow count, as appropriate. If the complex method does not improve the decision or explanation, prefer the simpler method. Record both outputs so later reviewers can see what the added structure contributed and what it obscured.
Invite a reviewer who was not involved in design to challenge scope, definitions, denominators, boundary conditions and incentives. Ask what evidence is systematically absent, who bears the cost of error, and how the method could be gamed. Convert each credible challenge into a test, collection action or explicit unresolved limitation rather than a defensive footnote.
Assign an owner, correction route, retention rule and review date. Verify that correction reaches derived outputs and future decisions, then retire the method when its assumptions, authority or evidence base no longer hold.
Design correction and appeal
Choose one consequential workflow and state the decision, the people affected, the time boundary and the claim being tested. Record original evidence before interpretation so later review can distinguish source material from inference.
Apply the article’s method to one routine case and one case near a boundary. Show the calculation or rule path, identify missing information and explain why plausible alternatives were accepted or rejected.
Change one material assumption at a time. Record what remains stable, what reverses and what becomes unresolved. Missing evidence must remain visible rather than being converted into a convenient zero, default or certainty.
Compare the result with at least one simpler alternative: explicit-only reading, an unweighted description, a transparent rule or a basic case-flow count, as appropriate. If the complex method does not improve the decision or explanation, prefer the simpler method. Record both outputs so later reviewers can see what the added structure contributed and what it obscured.
Invite a reviewer who was not involved in design to challenge scope, definitions, denominators, boundary conditions and incentives. Ask what evidence is systematically absent, who bears the cost of error, and how the method could be gamed. Convert each credible challenge into a test, collection action or explicit unresolved limitation rather than a defensive footnote.
Assign an owner, correction route, retention rule and review date. Verify that correction reaches derived outputs and future decisions, then retire the method when its assumptions, authority or evidence base no longer hold.
Set review and retirement triggers
Choose one consequential workflow and state the decision, the people affected, the time boundary and the claim being tested. Record original evidence before interpretation so later review can distinguish source material from inference.
Apply the article’s method to one routine case and one case near a boundary. Show the calculation or rule path, identify missing information and explain why plausible alternatives were accepted or rejected.
Change one material assumption at a time. Record what remains stable, what reverses and what becomes unresolved. Missing evidence must remain visible rather than being converted into a convenient zero, default or certainty.
Compare the result with at least one simpler alternative: explicit-only reading, an unweighted description, a transparent rule or a basic case-flow count, as appropriate. If the complex method does not improve the decision or explanation, prefer the simpler method. Record both outputs so later reviewers can see what the added structure contributed and what it obscured.
Invite a reviewer who was not involved in design to challenge scope, definitions, denominators, boundary conditions and incentives. Ask what evidence is systematically absent, who bears the cost of error, and how the method could be gamed. Convert each credible challenge into a test, collection action or explicit unresolved limitation rather than a defensive footnote.
Assign an owner, correction route, retention rule and review date. Verify that correction reaches derived outputs and future decisions, then retire the method when its assumptions, authority or evidence base no longer hold.
Questions readers ask
What is the first control?
- Define the population, domain or case boundary.
Why keep an unresolved state?
- Missing evidence must not become false certainty.
Does a formal method remove judgement?
- No.
What should be reviewed?
- Definitions, evidence, assumptions, outcomes and incentives.
When should the method change?
- When its evidence, authority or boundary conditions change.
What proves transfer?
- A changed-context test with the same reasoning structure.
Practise, verify and continue
Choose one consequential example, reproduce every step, test the strongest alternative explanation and write the condition that would change the result.
Apply the framework to an education decision. Preserve the reasoning structure while replacing domain authority, privacy, evidence and remedy.
Related eduKate reading
Uninorms, neutral elements and associativity limits
Sugeno integrals and non-compensatory choice
Bipolar Sugeno integrals and qualitative gains and losses
Choquet integrals and non-additive expectations
Authoritative sources and scope
Research on generalised aggregation properties
Weak dominance in aggregation operators
Triangular norms and conorms on ordered families
Continuous t-norm operations and aggregation
This article is educational. Examples are original composites. Definitions, standards and law can evolve; verify the cited primary sources and the current local context. A formal result supports only the named conclusion under stated assumptions. It does not establish truth, fairness, legality or fitness for every decision.