eduKateSG · Judgement · The Gold Standard of Education
Find the assumption that can overturn the answer before reality does
Choose the explanation you need, or follow all twelve chapters in order.
Full chapter index · Independent practice · Judgement subject guide
A decision can look precise because its options were scored, costs were totalled and a preferred choice appeared at the top. Yet the ranking may depend on one uncertain assumption: future demand, travel burden, learning fit, implementation time or the weight assigned to flexibility. If a small plausible change reverses the choice, the decision is fragile.
Robust judgement does not demand an option that wins under every imaginable future. It asks whether the choice remains reasonable across the uncertainties that matter, whether a reversing assumption is plausible, and whether the plan includes monitoring, adaptation or an affordable exit.
These chapters distinguish sensitivity from probability, calculate switching values, test several assumptions together and connect robustness with reversibility. The worked course-choice example is fictional. It teaches method rather than recommending any school, programme, investment, medical action or public policy.
Continue from The Gold Standard of Judgement within The Gold Standard of Education: learning that transfers into life and the How X Works learning library.
Full chapter index
See the mechanism · Chapters 1–2
Build the worked model · Chapters 3–5
Test what changes · Chapters 6–8
Check evidence and limits · Chapters 9–10
Practise and continue · Chapters 11–12
CHAPTER 1 OF 12 · See the mechanism
1. State the decision, objective and available options
Begin with the action, not the spreadsheet. “Which option is best?” is incomplete. State what the decision must achieve, for whom, over what period and within which constraints. Options are meaningful only relative to that purpose.
Include the current approach when it is feasible. A proposal often looks attractive because it is compared only with weaker alternatives, while the costs and benefits of continuing are left vague. The status quo is still an option with risks and consequences.
Separate eligibility or essential conditions from preferences. If a learner lacks a required prerequisite, a high score on convenience cannot make an unavailable programme feasible. Apply hard constraints before weighted comparisons.
Name who decides and who is affected. A student, family, tutor and institution may contribute different evidence and hold different authority. Robust analysis does not erase those roles.
Fix the decision date and review horizon. Some information may arrive soon enough to justify waiting; other delay carries a real cost. “Gather more evidence” is not automatically cautious if the opportunity expires.
The root guide, The Gold Standard of Judgement, separates evidence, assumptions, options and consequences. This specialist branch asks what happens when the assumptions move.
A decision model can be a simple table or a detailed simulation. In either case, list inputs, rules and outputs. Another reader should be able to identify where each conclusion comes from.
Mark observed values, forecasts, preferences and constraints differently. An observed travel time from three journeys is not the same kind of input as a forecast of next year’s workload or a family preference for schedule flexibility.
Define the scoring direction. Does a higher number always mean better? If a burden is subtracted while a benefit is added, show the signs. Hidden direction errors can reverse a ranking before sensitivity analysis even begins.
Avoid false precision. A learning-fit score of 7.43 may imply measurement that the evidence cannot support. Coarser categories with clear reasons can be more honest and easier to challenge.
Trace each value to evidence or a named judgement. “Option B costs 2 points” should link to the travel estimate, weighting rule and decision owner. An unexplained number cannot be stress-tested meaningfully.
Finally, keep narrative reasons beside totals. A model organises judgement; it does not replace it. If two options tie, the accepted trade-off and implementation conditions still matter.
CHAPTER 3 OF 12 · Build the worked model
3. Identify the assumptions that could change the choice
An assumption matters when changing it alters a comparison or action. Start with uncertain inputs that have a direct path to the result: costs, uptake, completion, timing, effect size, demand, reliability or preference weights.
Ask each participant, “What would have to be true for your preferred option to disappoint?” This question can reveal assumptions hidden behind confidence. It is less confrontational than asking someone to defend the whole choice at once.
Distinguish structural assumptions from numerical values. “Travel time affects attendance” is structural; “each additional weekly hour costs two points” is numerical. Both can be tested, but they require different evidence.
Look for asymmetry. A small downside in the model may carry a large real consequence for one group. Distributional effects should not disappear inside an average total.
Rank assumptions by influence and uncertainty. A highly uncertain input that barely changes the result deserves less attention than a moderately uncertain input near a switching point. This keeps analysis proportionate.
Write an assumption register: statement, evidence, plausible range, owner, effect on the decision and review trigger. The register turns “we considered uncertainty” into something another person can inspect.
The fictional table scores three options on learning fit, travel burden and flexibility. The initial totals are A = 19, B = 21 and C = 21. A quick reading calls B and C joint leaders.
But the totals hide how the numbers were combined. Suppose travel burden is already expressed as a positive convenience score. If a reviewer mistakenly treats it as a penalty and subtracts it, the ranking changes. Define meaning before arithmetic.
Assume the values are additive for teaching purposes. Option B has strong travel convenience but lower flexibility than C. Option C leads on flexibility but has the weakest learning-fit score. The tie reflects different strengths, not identical options.
Now ask whether a one-point difference is meaningful given the evidence. If scores were elicited informally, distinctions of one or two points may be within judgement noise. A robust conclusion may be “B and C are both plausible; gather evidence about the decisive trade-off.”
Option A should not be discarded automatically. Its learning fit is highest. If learning fit receives greater weight, or travel circumstances improve, A may become preferred. Sensitivity analysis shows this route.
The table is not an objective machine. It makes priorities visible. The strongest output is an explanation of which assumptions sustain each option and what evidence could separate them.
| Option | Learning fit | Travel burden | Flexibility | Initial total |
|---|---|---|---|---|
| A | 8 | 4 | 7 | 19 |
| B | 7 | 8 | 6 | 21 |
| C | 6 | 6 | 9 | 21 |
CHAPTER 5 OF 12 · Build the worked model
5. Distinguish sensitivity, uncertainty and probability
Sensitivity asks how much an output changes when an input changes. Uncertainty describes what is not known precisely. Probability assigns a quantified likelihood under a model. A sensitivity test can be useful even when reliable probabilities are unavailable.
Change one input across a plausible range and recalculate. A steep response means the result is sensitive. That does not mean the extreme value is likely; it means the assumption deserves attention if the range is credible.
Probability analysis goes further by assigning distributions and examining outcome frequencies. It can be valuable when data and modelling support it, but invented probabilities create a polished fiction.
Scenario analysis changes several connected assumptions together. High demand may coincide with longer waiting times and staffing pressure. Varying each independently can produce combinations that are internally inconsistent.
Deep uncertainty arises when parties do not agree on models, distributions or important futures. Robust approaches may then compare how options perform across many plausible cases rather than optimise for one forecast.
Use language that matches the method: “the ranking is sensitive to travel burden” is not “there is a 70% chance Option A wins.” The first can be true without evidence for the second.
CHAPTER 6 OF 12 · Test what changes
6. Calculate a switching value instead of waving at uncertainty
A switching value is the input value at which the preferred option changes or a proposal ceases to meet a criterion. It converts a vague concern into a threshold that can be compared with evidence.
Suppose Option B leads A by two points under the current assumptions. If increasing B’s travel penalty by 2.1 points makes A preferred, 2.1 is the approximate switching increment. The next question is whether such a penalty is plausible.
The UK Green Book 2026 describes sensitivity analysis and switching values in public appraisal. Its formal value-for-money setting is different from the fictional learning example; the transferable idea is to expose the value at which the decision changes.
Solve algebraically when the model is simple. If net value of A is fixed at 19 and B is 23 minus travel penalty x, the tie occurs when 23 − x = 19, so x = 4. A penalty above four makes A lead under that model.
For nonlinear models, use a grid or root-finding method and verify the result. Keep units and direction visible. A threshold without a defined input is useless.
Report the baseline, switching value and evidence range together. A switch far outside plausible values supports robustness. A switch inside the observed range signals a decision that may need more evidence, adaptation or a different criterion.
The fictional table states that B’s weekly travel penalty has a base value of two points and a switching value of 4.1. If realistic evidence places the penalty between one and three, B’s position is robust to this uncertainty. If journeys frequently produce values above five, the initial ranking is fragile.
For C, the value assigned to flexibility starts at three points and switches at 1.2. The wide margin suggests C remains attractive unless flexibility matters much less than initially judged. But the preference belongs to the decision context, not a universal scale.
A’s learning-fit score starts at eight and loses competitiveness below 6.8. The useful question is whether the evidence can distinguish eight from 6.8. If the score came from one brief observation, the margin may be less reassuring than it looks.
Switching values can reveal where research is worth doing. Measuring an input far from its switch may not change the decision. Investigating a weakly evidenced input near its switch can have high value.
Do not test only the winning option. An alternative may become attractive under a plausible assumption but face a hard implementation barrier. Robust judgement combines performance thresholds with feasibility.
The decision record should say: preferred option under base assumptions, nearest credible switching condition, evidence about that condition, and planned response if it occurs.
| Assumption tested | Base value | Switching value | Interpretation |
|---|---|---|---|
| Weekly travel penalty for B | 2 points | 4.1 points | Above 4.1, A becomes preferred |
| Value of flexibility for C | 3 points | 1.2 points | Below 1.2, C loses its lead |
| A learning-fit score | 8 | 6.8 | Below 6.8, A is no longer competitive |
CHAPTER 8 OF 12 · Test what changes
8. Change several assumptions together with scenarios
Construct a small set of coherent scenarios rather than every possible combination. For a course choice, examples might be stable timetable, heavier CCA commitment, improved transport and changed learning needs. Each scenario should have a clear rationale.
Recalculate all options under the same scenario. Do not give the preferred option optimistic inputs and competitors conservative ones. Symmetric treatment is part of fair comparison.
Look for dominance. If one option performs at least as well across every credible scenario and better in some, it has a strong robustness case. More often, options trade places.
When rankings change, identify the scenario feature causing the switch. This turns a large scenario story into a monitorable condition such as weekly travel exceeding a threshold or a prerequisite remaining unmet.
Avoid averaging scenarios blindly. A high average score can hide failure in a scenario with unacceptable consequences. Apply minimum standards before comparing averages.
Use the existing Model Discrimination guide to choose observations that make competing explanations or plans predict differently. The best next evidence separates options, not merely adds more data.
CHAPTER 9 OF 12 · Check evidence and limits
9. Connect robustness with reversibility and staged action
A fragile decision is less dangerous when it is cheap to reverse and monitored early. A robust plan may therefore be a sequence: try a limited action, observe, then expand, revise or stop.
Define exit costs. Financial cost is only one part; switching can affect time, confidence, relationships, learning continuity or public trust. State who bears those costs.
Use pilots when they can genuinely test the decisive assumption. A pilot that measures activity but not the outcome driving the choice creates motion without information.
Set a review date and trigger before implementation. “Review if it goes badly” invites hindsight. “Review after four weeks if independent method selection remains below the agreed criterion” is observable.
Preserve optionality selectively. Keeping every option open can consume resources and delay learning. Maintain the options whose value under plausible change justifies their carrying cost.
For irreversible or high-consequence choices, demand stronger evidence and wider stress tests. Robustness is proportionate to consequence, not a ritual applied equally to every everyday preference.
CHAPTER 10 OF 12 · Check evidence and limits
10. Recognise limits and common robustness misconceptions
First misconception: robust means optimal in every scenario. Usually it means acceptable across a relevant range, with transparent trade-offs. No option can dominate every imaginable future.
Second misconception: more scenarios always improve judgement. Redundant or fantastical scenarios can distract from the uncertainties that drive the decision. Use a small, meaningful set and explain why each matters.
Third misconception: a switching value predicts what will happen. It identifies a threshold. Evidence is still needed to judge whether the threshold is plausible or approaching.
Fourth misconception: sensitivity analysis repairs a bad model. If essential options are missing, criteria are misdefined or evidence is biased, varying numbers inside the model cannot make the structure sound.
Fifth misconception: a stable ranking proves the chosen option will succeed. All options may perform poorly under some conditions. Test absolute acceptability as well as relative rank.
Finally, robustness does not settle ethical or value conflicts automatically. Evidence can show consequences and thresholds; legitimate decision-makers still have to explain priorities, rights and distributional effects.
CHAPTER 11 OF 12 · Practise and continue
11. Practise with a one-page robustness record
Choose a manageable decision with two or three feasible options. Write the objective, hard constraints and three decision criteria. Give each input a source or label it as a judgement.
Calculate the base ranking. Then identify the uncertain assumption with the clearest path to the result. Move it until the preferred option changes. Record the switching value and compare it with a plausible evidence range.
Build two coherent scenarios by changing connected assumptions together. Note whether any option fails a minimum condition even when its total remains high.
Design one observation that could update the decisive assumption. Set a date, owner and action. A robustness analysis without a monitoring route can become a static report.
Questions learners often ask Should every assumption be tested? Focus on influential and uncertain ones. Is a tie a failure? No; it reveals that narrative trade-offs or more evidence matter. Can preferences be tested? Their consequences can be explored, though values are not mere factual errors. Does robust mean cautious? It can support decisive action when an option survives meaningful stress.
Finish with a bounded statement: “We choose ___ because it remains acceptable across ___; the choice reverses if ___ reaches ___; we will monitor ___ and review by ___.”
CHAPTER 12 OF 12 · Practise and continue
12. Connect robust choice to the wider Judgement route
Return to The Gold Standard of Judgement for decision definition, evidence, authority, consequences and post-decision review. Robustness extends the root by making assumption change visible.
Read Evidence-Informed Decision Making to connect research, local data, professional judgement and values. A switching threshold is only useful when the evidence feeding it is relevant and traceable.
Use Post-Decision Evidence Integration when new information arrives after a choice. Monitoring should update the model without forcing every surprise into the original story.
The UK Green Book 2026 and its sensitivity guidance were checked on 22 October 2026. They support the use of sensitivity analysis and switching values in formal appraisal. The scores, thresholds and learning-choice examples here are original and deliberately simplified.
This article does not replace regulated appraisal, financial advice, clinical decisions, legal duties or institution-specific admissions rules. High-consequence choices require the applicable authority, methods and expertise.
The durable transfer is to stop asking only, “Which option has the highest total?” Ask, “Which assumption holds it there, how far can that assumption move, and what will we do before it crosses the line?”
A final robustness review before commitment First, ask whether the shortlisted options are genuinely different. Three labels can hide the same underlying strategy, while a meaningful alternative remains excluded. Describe the mechanism by which each option is supposed to achieve the objective.
Second, challenge the baseline. Historical averages may not fit a changed environment, and the latest observation may be an outlier. Show why the selected baseline is relevant to the decision horizon. When two baselines are defensible, test both.
Third, distinguish controllable choices from external conditions. Implementation quality, staffing and communication may be partly controllable; demand, weather or policy change may not be. Monitoring and contingency plans should reflect who can act on each variable.
Fourth, inspect dependencies. The success of one option may rely on another project finishing first, a partner maintaining service or a learner sustaining a schedule. Correlated assumptions can fail together, so independent one-variable tests may understate exposure.
Fifth, test downside concentration. A modest expected loss spread across many people differs from a severe loss concentrated on a small group. A total score should not conceal minimum protections, rights or unacceptable outcomes.
Sixth, identify information that will arrive naturally. If a decisive result is due next week and delay is cheap, waiting may be sensible. If evidence will arrive only after commitment, build a staged or reversible action instead.
Seventh, examine incentives around the model. The person who benefits from approval may estimate costs optimistically; the person carrying implementation risk may emphasise obstacles. This does not invalidate either view, but independent evidence and transparent ranges become especially valuable.
Eighth, pre-mortem the preferred option. Imagine the decision disappointed after one year. List plausible mechanisms, then ask whether the current model represents them. Use the exercise to find missing assumptions, not to invent certainty about failure.
Ninth, conduct a reverse stress test. Instead of asking how the option behaves under chosen scenarios, ask what combination of conditions would make it unacceptable. Compare that boundary with evidence and monitoring capability.
Tenth, preserve the rejected alternatives and reasons. If conditions change, the group can revisit a live option without reconstructing the entire debate. Record which assumption would make each alternative worth reopening.
Eleventh, assign ownership. A threshold nobody monitors is decorative. Name the person or system responsible for each trigger, where the data will come from and who has authority to pause, adapt or reverse the action.
Twelfth, schedule a process review as well as an outcome review. Ask whether evidence was gathered fairly, whether thresholds were followed and whether unanticipated effects appeared. A favourable outcome can still reveal a fragile process that should be improved before the next decision.
When reasonable people still choose differently Two decision-makers can share facts and still choose differently because their objectives, risk tolerances, time horizons or distributional priorities differ. Robust analysis should expose this value difference rather than manufacture agreement through arbitrary weights.
Ask each person to specify an unacceptable outcome and a preferred trade-off. One family may prioritise daily travel stability; another may accept travel for a distinctive learning fit. Neither preference becomes a fact, but each can be applied consistently across options.
Then test whether disagreements are material. If every plausible preference set selects the same option, debate over small weights may not matter. If modest value changes switch the result, the final explanation should acknowledge that reasonable priorities lead to different choices.
For shared decisions, use an agreed authority and escalation route. Consultation can gather evidence and values without pretending that everyone holds the same formal responsibility. Explain how dissent was considered and which protection remains for those bearing downside risk.
Avoid retrospective weight changes. Once the preferred option is visible, altering weights to make it win converts analysis into advocacy. If the model genuinely omitted a criterion, document the correction and rerun every option under the revised rule.
Finally, preserve humility without paralysis. A decision can be justified, monitored and revisable even when uncertainty and value conflict remain. Robust judgement is the discipline of acting with visible assumptions, not the fantasy of removing every reason to disagree.
Keep the arithmetic reproducible Save the base inputs, formulas, scenario changes and outputs in a versioned worksheet or notebook suited to the task. Lock labels and units so a later reviewer can distinguish a percentage from a proportion, annual cost from total cost and benefit from penalty.
Test the model with hand-calculated cases. A formula copied into the wrong row can make sensitivity charts internally consistent and still wrong. Reproduce at least one base score and one switching threshold independently.
When the model changes, compare versions. Identify which ranking changes came from new evidence, which came from corrected arithmetic and which came from revised values. This prevents a genuine correction from being confused with a change of preference.
Publish a compact reader-facing explanation rather than every intermediate cell. The evidence record can remain detailed while the decision note states the objective, preferred option, decisive assumptions, switching conditions and monitoring plan. Transparency means traceability, not overwhelming the audience.
Archive the decision-time model before outcomes arrive. Later knowledge makes some events look obvious and invites silent rewriting of earlier expectations. A dated record allows the review to ask whether the original process was reasonable, which assumptions failed and what a future decision should change.