The Gold Standard of Judgement · Specialist guide
How to plan now, learn over time, and make later decisions only with information that would actually be available.
The transfer test asks whether the reasoning still works when the system, evidence, timing or affected group changes.
Chapter contents
Foundations
Model and method
Implementation
Chapter 1 of 12
A plan can be a policy, not one answer
Many decisions unfold in stages. A school books rooms now, updates staffing after enrolment, and reallocates support after early assessments. A single fixed plan ignores information that will arrive.
Multistage stochastic programming chooses initial actions and contingent later actions for possible histories. The result is a policy: what to do after each observable state, not a clairvoyant schedule.
The gold standard matches decision timing to information timing and makes the cost of flexibility visible.
Practical check
- Decision stages
- Uncertainty revealed between stages
- Actions available at each stage
- State carried forward
- Outcome and risk measures
Chapter 2 of 12
Here-and-now and recourse decisions
Here-and-now decisions are taken before uncertainty is known and often cannot be reversed cheaply. Recourse decisions adapt after some uncertainty is observed.
Good models distinguish reversible, partly reversible and irreversible actions. Hiring permanent staff differs from assigning overtime; building capacity differs from renting it.
If every decision is allowed to adapt freely, the model overvalues flexibility. If none can adapt, it may recommend excessive buffers.
| Decision | Timing | Typical flexibility |
|---|---|---|
| Reserve budget | Before demand | Fixed envelope |
| Hire temporary staff | After early signal | Limited recourse |
| Reassign rooms | After enrolment | Operational recourse |
| Cancel programme | Late stage | High-cost recourse |
Chapter 3 of 12
Stages follow information, not the calendar alone
A stage is defined by what is known when a decision is made. Two calendar dates with the same information can be one stage; two decisions on the same day may be different stages if one follows a measurement.
Write an information timeline before equations. Identify forecast updates, observations, delays and decisions that cannot wait.
Bad stage design permits impossible foresight or withholds information the real organisation would use.
Practical check
- Observation time
- Decision deadline
- Data latency and quality
- What remains unknown
- What actions remain feasible
Chapter 4 of 12
Scenario trees represent shared histories
A scenario tree branches as uncertainty is revealed. Nodes with the same path from the root share the same history and therefore the same information.
Branches need probabilities or weights, state transitions and outcomes. A tree is an approximation, not a catalogue of every possible future.
A useful tree preserves the dependencies and decision-relevant variation that shape policy. More branches are not automatically more realistic.
Practical check
- Root and stage structure
- Branch probabilities
- State transition consistency
- Dependence across variables
- Decision-relevant extremes
Chapter 5 of 12
Worked example: tutoring capacity over a term
A centre commits to six core classes before enrolment. After registration, demand may be low, medium or high. After the first fortnight, attendance may persist or fall, allowing room and teacher reassignment.
A two-stage model can add temporary capacity after registration but cannot adapt again after attendance evidence. A three-stage model can reserve rooms initially, add sessions after demand, then consolidate or extend after persistence is observed.
The policy should not choose the high-demand branch’s room allocation while still at the root. That would use future enrolment illegally.
| Stage | Known information | Permitted decision |
|---|---|---|
| 0 | Forecast only | Reserve rooms and core staff |
| 1 | Registration level | Add temporary sessions |
| 2 | Attendance persistence | Consolidate, extend or reassign |
Chapter 6 of 12
Non-anticipativity prevents foresight
Non-anticipativity requires decisions to be identical for scenarios that share the same observed history. Before branches separate, the policy cannot react differently to them.
Violations often hide in spreadsheets or decomposed models where each scenario is optimised independently. Those solutions look excellent because they know the future.
Check every decision against its information set. A variable indexed by full scenario can be dangerous when only the current node is known.
Practical check
- Same history, same decision
- No use of later outcomes in earlier features
- Forecast origin respected
- Scenario-specific variables linked at common nodes
- Policy executable from observed state
Chapter 7 of 12
Recourse has limits and costs
Recourse may be unavailable, delayed or expensive. Emergency procurement, overtime and cancellation solve different problems and create different consequences.
Model capacity, lead time, fixed costs and fatigue or quality effects. Unlimited cheap recourse makes early planning look unnecessarily cautious.
Some failures cannot be repaired later. Safety floors, legal duties and lost instructional time may need hard first-stage protection.
Practical check
- Recourse trigger
- Lead time and capacity
- Direct and indirect cost
- Reversibility
- Non-recoverable harm
Chapter 8 of 12
Probabilities and tree generation
Scenario probabilities may come from forecasts, historical transitions, physical models or structured judgement. They should reflect conditional paths rather than independent draws pasted together.
Tree reduction can make computation possible but may remove rare pathways that drive protective decisions. Preserve decision-relevant tails and explain what was compressed.
Version the tree, forecast origin and generation method. A policy cannot be reproduced from branch labels alone.
Practical check
- Data and forecast origin
- Conditional dependence
- Rare-event treatment
- Reduction or clustering method
- Version and random seed
Chapter 9 of 12
Objectives can include risk over time
Minimising expected cost can tolerate severe loss on a small branch. Add risk measures, service floors or constraints when tail outcomes matter.
Risk preferences should be time-consistent enough that later actions do not contradict the policy solely because the evaluation rule changes.
Show the expected result, tail result and worst credible pathways separately. A single objective value hides policy shape.
Practical check
- Expected performance
- Tail or downside measure
- Hard floors
- Stagewise risk treatment
- Group and consequence distribution
Chapter 10 of 12
Value of adaptability and information
Compare a fixed policy, a two-stage policy and a multistage policy under the same uncertainty. The performance difference estimates the value of adaptability under the model.
Also compare perfect-information and current-information bounds. A huge gap may reveal valuable measurement, but only if the information can arrive before the decision and change an action.
Flexibility has operational cost. A theoretical gain that requires impossible monitoring or rapid response is not real value.
Practical check
- Fixed versus adaptive benchmark
- Perfect-information bound
- Observation cost and latency
- Implementation capability
- Sensitivity to forecast error
Chapter 11 of 12
Validate policies out of sample
Test the policy on paths not used to build the tree. Simulate the information sequence and execute only actions available at each stage.
Measure cost, feasibility, recourse frequency, tail harm and instability. Review whether small forecast changes cause wild action shifts.
Use rolling backtests where forecasts are recreated from information available at each historical decision date. Retrospective full data can leak future knowledge.
Practical check
- Unseen scenario paths
- Stage-by-stage execution
- No future-data leakage
- Tail and constraint outcomes
- Policy stability and override rate
Chapter 12 of 12
Transfer check: plan a school event
Book a venue before weather is known, choose indoor or outdoor configuration after a forecast, then adjust staffing after attendance confirmation. Draw the information timeline and tree.
Change the context: the forecast arrives after the venue deadline. The earlier decision can no longer depend on it, so non-anticipativity changes the feasible policy.
A reviewer should reproduce branch probabilities, shared-node decisions, recourse limits and validation. If the plan cannot be executed without knowing the branch label, it is not a policy.
Practical check
- Map observation and action times
- Identify irreversible commitments
- Build shared histories
- Enforce non-anticipativity
- Test on new paths
Advanced cases and implementation boundaries
Node-based versus scenario-based models
Node variables naturally encode shared history; scenario variables can be easier for decomposition but require explicit non-anticipativity links. Choose a formulation whose information structure can be audited.
Test a small instance both ways. Matching objective and decisions at common nodes gives confidence that the links are correct.
Endogenous uncertainty
Some actions change what will be learned or the distribution itself. A pilot programme reveals demand; an inspection changes compliance; capacity changes customer behaviour.
Standard trees often assume uncertainty is external. Model decision-dependent observation carefully or state the limitation and stress test responses.
Partial observation
The true state may remain hidden while noisy signals arrive. Decisions should depend on the observed signal or belief state, not the unobserved reality.
Record measurement error and update rule. A model that branches on true demand before it is measured anticipates information.
Commitment and consistency
Later optimisation may prefer abandoning a promise made earlier. Contracts, fairness and reputation can require commitment constraints or penalties.
Identify which decisions may be revised and who bears the change. Adaptability without governance can shift uncertainty onto others.
Nested risk
A policy evaluated only by final total cost can hide unacceptable interim exposure. Stagewise or nested risk measures can keep risk treatment coherent through time.
Explain how risk is updated after observations. A rule that reverses preferences solely because time passed is difficult to govern.
Probability ambiguity
Tree probabilities are estimates. Re-solve under alternative transition matrices and inspect which actions remain stable.
If small probability changes reverse early commitments, the decision needs more evidence, flexibility or a robust safeguard.
Scenario reduction
Reduction should preserve distances or features related to decisions, not only overall distribution. A rare branch may matter because recourse fails there.
Compare reduced and richer trees on an independent simulation. Report lost tail behaviour and action changes.
Decomposition and convergence
Large models may be solved with decomposition or sampling. Stopping early yields an approximate policy whose quality needs bounds or empirical evidence.
Store solver settings, convergence diagnostics and random seeds. Reproducibility includes computation, not just algebra.
Governance of forecasts
A forecast owner, model version, refresh time and error monitoring should accompany the optimisation. The decision model must know which forecast vintage it consumed.
When the forecast changes, determine whether policy reapproval is required. Automatic re-solving can alter commitments invisibly.
Evidence packet
Retain formulation, tree, probabilities, data, code, solver log, policy table, backtest and approval of risk and recourse assumptions.
A reviewer should replay one complete path and verify that each decision used only information available at that node.
Chapter-by-chapter evidence audit
This audit turns the teaching chapters into a repeatable review. It is intentionally concrete: every conclusion should connect to a record, boundary test and accountable next step.
Audit 1: A plan can be a policy, not one answer
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Decision stages; Uncertainty revealed between stages; Actions available at each stage; State carried forward; Outcome and risk measures. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 2: Here-and-now and recourse decisions
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: . Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 3: Stages follow information, not the calendar alone
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Observation time; Decision deadline; Data latency and quality; What remains unknown; What actions remain feasible. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 4: Scenario trees represent shared histories
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Root and stage structure; Branch probabilities; State transition consistency; Dependence across variables; Decision-relevant extremes. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 5: Worked example: tutoring capacity over a term
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: . Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 6: Non-anticipativity prevents foresight
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Same history, same decision; No use of later outcomes in earlier features; Forecast origin respected; Scenario-specific variables linked at common nodes; Policy executable from observed state. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 7: Recourse has limits and costs
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Recourse trigger; Lead time and capacity; Direct and indirect cost; Reversibility; Non-recoverable harm. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 8: Probabilities and tree generation
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Data and forecast origin; Conditional dependence; Rare-event treatment; Reduction or clustering method; Version and random seed. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 9: Objectives can include risk over time
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Expected performance; Tail or downside measure; Hard floors; Stagewise risk treatment; Group and consequence distribution. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 10: Value of adaptability and information
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Fixed versus adaptive benchmark; Perfect-information bound; Observation cost and latency; Implementation capability; Sensitivity to forecast error. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 11: Validate policies out of sample
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Unseen scenario paths; Stage-by-stage execution; No future-data leakage; Tail and constraint outcomes; Policy stability and override rate. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Audit 12: Transfer check: plan a school event
A warning sign is that the recommended action depends on information that would arrive only after the real decision deadline. Review this chapter through the concrete checks already defined: Map observation and action times; Identify irreversible commitments; Build shared histories; Enforce non-anticipativity; Test on new paths. Record which check is satisfied, which remains uncertain and who owns the next action.
Retain one worked record showing the input, governing definition or information set, transformation or decision, result and reviewer. Change one material condition and repeat the test. If the conclusion changes, explain whether that sensitivity is intended, evidence-led and acceptable.
Field workbook: turn the idea into evidence
Use these prompts to produce a reviewable specification, worked test and approval record. Each step should leave evidence another person can inspect without relying on private memory.
Policy table
Translate the solution into a table that operators can use: observed state, permitted information, decision, resource requirement and exception path. If implementation requires knowing a scenario label, the model failed.
Keep the table small enough to execute or provide a tested decision rule. A huge scenario-specific plan can be mathematically valid but operationally useless.
State design
State variables carry information forward: remaining budget, capacity, inventory, commitments or unresolved demand. They should be sufficient for later decisions without retaining irrelevant history.
An incomplete state creates inconsistent policies; an oversized state makes computation and governance difficult. Test whether two histories with the same state truly allow the same future choices.
Rolling horizon use
In practice, organisations often re-solve as new data arrives. Rolling horizons can add realism but risk changing commitments or objectives unexpectedly.
Lock irreversible decisions, preserve previous promises and record each forecast and policy version. Re-optimisation is not permission to forget the past.
Scenario stress overlay
Probability-weighted trees can underrepresent rare disruption. Add labelled stress branches or separate robust tests, but do not mix stress weights into ordinary frequencies without explanation.
Ask whether the policy has a safe degraded mode outside the tree. Recourse designed only for modelled variation may collapse under structural change.
Computational approximation
Large trees grow rapidly with stages and branching. Decomposition, sampling, aggregation and tree reduction make problems solvable but can change policy quality.
Report optimality gaps or bounds when available, and compare action stability under alternative approximations. Computation is part of uncertainty.
Human override
Define which observations allow operators to depart from the policy, who authorises the change and how consequences are recorded. Overrides can protect against model error but also destroy consistency.
Review patterns of override. Repeated departures may reveal missing state, poor probabilities or unrealistic recourse assumptions.
Practice task
Build a three-stage inventory tree: order now, expedite after an early demand signal, and reallocate after regional demand is observed. Enforce equal root decisions across all paths.
Then delay the early signal until after the expedite deadline. Remove the unavailable adaptation and compare cost, service and buffer.
Approval card
Approve only when stages, information sets, state transitions, probabilities, recourse, non-anticipativity, risk, validation and implementation are documented.
Require one path replay using only contemporaneous information. If the action cannot be reproduced, the policy is not auditable.
Reading routes across the eduKate ecosystem
Begin with The Gold Standard of Judgement. Place this specialist guide inside The Gold Standard of Education: Learning Transfers into Life, then use the How X Works hub to compare methods across domains.
Related specialist guides
Frequently asked questions
How is multistage different from two-stage programming?
Two-stage models allow one round of recourse after uncertainty. Multistage models allow several observation-and-decision cycles.
What is non-anticipativity?
Scenarios with the same information history must receive the same decision until their histories diverge.
Is a scenario tree a forecast?
It is a structured approximation of possible conditional paths used for decisions. It is not a claim that those are the only futures.
Why not optimise every scenario separately?
That gives each scenario knowledge of its own future and produces a clairvoyant, non-executable solution.
Does more recourse always improve the plan?
Only in the model. Real recourse has cost, capacity, latency and sometimes cannot repair the harm.
How should the policy be validated?
Replay unseen paths using only information available at each stage and measure both average and tail outcomes.
Authoritative sources
- Heitsch and Römisch Scenario tree modelling for multistage stochastic programs
- Heitsch and Römisch Scenario tree generation for multistage stochastic programs
- Dentcheva and Ruszczynski Subregular Recourse in Nonlinear Multistage Stochastic Optimization
- Carnegie Mellon University Models and Computational Strategies for Multistage Stochastic Programming
Source links support the technical or institutional foundations. The worked examples and transfer checks are original teaching constructions, not reports of real people or cases.