Make the decision path visible
A Pareto frontier displays defensible compromises but does not choose among them. This guide shows how explicit loss functions, thresholds and regret can turn a frontier into a decision without disguising values as facts.
A Pareto frontier displays defensible compromises but does not choose among them. This guide shows how explicit loss functions, thresholds and regret can turn a frontier into a decision without disguising values as facts. The aim is not to force certainty where the evidence is conditional. It is to show what can be used now, what needs further testing and what must remain visible to the next person.
The chapters move from first principles to a worked case, changed-context transfer, explicit limits, practical checks and an auditable decision. Start with the route closest to your task or read in sequence.
Choose the route closest to your decision
Chapter contents
Route 1: chapters 1–3
Route 2: chapters 4–6
Route 3: chapters 7–9
1. Recognise that the frontier is not the decision
A Pareto frontier removes dominated options but usually leaves several incomparable choices. Selection requires preferences, loss, constraints or decision rights beyond dominance. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. What action must be chosen, by whom and by when? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. Three model repairs are nondominated: one drops a restriction, one adds slack and one widens the identified set. None is mathematically best on every criterion. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the distinction to treatment selection where efficacy, burden and uncertainty trade off. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. Calling every frontier point optimal can hide that the organisation still owes a choice. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Record the action, decision owner, deadline, admissible frontier and consequences of deferral. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
2. Define states, actions and consequences
Write the decision problem before writing the loss function: possible states, available actions and consequence measures must be interpretable in the subject domain. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Can every term in the loss be connected to a consequence rather than an analyst’s convenience? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. The state is the treatment effect, actions are adopt or wait, and consequences include missed benefit, avoidable harm and research delay. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the structure to a policy threshold with graded rather than binary actions. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. A loss on parameter distance may not represent the real cost of a wrong decision. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Keep state space, action set, outcome horizon, affected groups, consequence units and exclusions. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
3. Express loss in meaningful units
Whenever possible, define loss using outcome, resource, welfare or scientific-error units. Keep incomparable consequences separate until an authorised tradeoff is supplied. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. What does a one-unit increase mean to the people using the decision? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. A false adoption loss counts expected harmful outcomes, while a false rejection loss counts foregone beneficial outcomes; neither is replaced by an arbitrary squared parameter error. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the method to model repair by valuing incorrect scientific claims and unnecessary agnosticism separately. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. Normalising numbers to zero-to-one does not make their consequences commensurable. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Record units, time horizon, discounting, affected population, transformations and who authorised any exchange rate. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
| Checkpoint | What to test |
|---|---|
| Decision question | What does a one-unit increase mean to the people using the decision? |
| Worked comparison | A false adoption loss counts expected harmful outcomes, while a false rejection loss counts foregone beneficial outcomes; neither is replaced by an arbitrary squared parameter error. |
| Transfer test | Transfer the method to model repair by valuing incorrect scientific claims and unnecessary agnosticism separately. |
| Known boundary | Normalising numbers to zero-to-one does not make their consequences commensurable. |
4. Make thresholds and asymmetry visible
Decisions often change at clinical, policy or scientific thresholds, and errors on opposite sides need not cost the same. Encode those asymmetries openly. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Would a tiny estimate change near the threshold reverse the action and create a large loss jump? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. An intervention is adopted only above a benefit threshold; harm from false adoption is weighted more heavily for a vulnerable subgroup. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the threshold check to environmental action triggered by a contamination limit. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. A smooth estimation loss can conceal a discontinuous decision consequence. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Keep threshold source, tolerance band, asymmetric costs, subgroup rules and sensitivity to moving the boundary. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
5. Choose among expected loss, worst case and regret
Expected loss uses a probability distribution over states; worst-case loss protects against the most adverse state; regret compares the chosen action with the best action in hindsight. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Which uncertainty representation is justified, and what kind of error is the decision trying to avoid? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. A credible distribution supports expected-loss ranking, while a poorly identified model leads the team to compare maximum regret across frontier repairs. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the comparison to sequential studies where delay changes both evidence and cost. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. Minimax rules can be overly conservative, while expected loss can be overconfident when probabilities are fragile. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Record probability model, ambiguity set, regret benchmark, loss bounds and reasons for the selected criterion. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
| Checkpoint | What to test |
|---|---|
| Decision question | Which uncertainty representation is justified, and what kind of error is the decision trying to avoid? |
| Worked comparison | A credible distribution supports expected-loss ranking, while a poorly identified model leads the team to compare maximum regret across frontier repairs. |
| Transfer test | Transfer the comparison to sequential studies where delay changes both evidence and cost. |
| Known boundary | Minimax rules can be overly conservative, while expected loss can be overconfident when probabilities are fragile. |
6. Work a frontier-selection case
Calculate the decision consequences for every nondominated repair before selecting. Show how weights, thresholds and uncertainty change the ranking. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Would a plausible alternative loss choose another frontier point? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. Repair A gives a narrow positive estimate but requires dropping a safety restriction; B includes zero with smaller assumption loss; C is wider but avoids irreversible adoption. Low-regret analysis selects B and wait. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Change the harm ratio and show the point at which A or C becomes preferred. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. A unique numerical minimum can be unstable and should not be reported without the surrounding sensitivity. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Save the frontier table, consequence calculations, loss curves, switch points, uncertainty and chosen action. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
7. Elicit preferences without laundering them
Decision-makers may supply weights, thresholds or constraints, but the process should reveal whose values count and how conflicts are handled. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Can the final choice distinguish empirical evidence from authorised preference? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. Patients, clinicians and funders value delay and adverse events differently. The analysis reports each perspective before the accountable body chooses a rule. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the elicitation to public policy by including distributional impacts rather than only average loss. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. Consultation is not a licence to average incompatible ethical claims into one hidden number. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Keep participants, method, competing views, conflicts of interest, unresolved disagreement and final authority. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
8. Carry statistical uncertainty into loss and regret
Loss estimates depend on uncertain effects, costs and frontier membership. Propagate joint uncertainty rather than evaluating only point estimates. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Could sampling variation change which option minimises expected loss or regret? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. Bootstrap draws move two repairs on and off the frontier and reverse the preferred action in twenty percent of draws; the decision is labelled fragile. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the analysis to clustered data with a resampling plan that respects the design. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. Independent confidence intervals can miss correlation between costs and inferred effects. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Record sampling model, simulation, coverage target, numerical error, decision probabilities and instability warnings. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
| Checkpoint | What to test |
|---|---|
| Decision question | Could sampling variation change which option minimises expected loss or regret? |
| Worked comparison | Bootstrap draws move two repairs on and off the frontier and reverse the preferred action in twenty percent of draws; the decision is labelled fragile. |
| Transfer test | Transfer the analysis to clustered data with a resampling plan that respects the design. |
| Known boundary | Independent confidence intervals can miss correlation between costs and inferred effects. |
9. Stress-test ambiguity and model misspecification
Vary probability distributions, loss ratios, thresholds, frontier approximations and admissible repairs. Look for decisions that remain acceptable across the plausible set. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Which assumptions are decision-critical rather than merely statistically noticeable? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. The preferred repair changes under modest tail-risk assumptions, but the wait action remains below a regret cap across all scenarios. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the test to distribution shift between trial and target population. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. A robust action may still be poor if the scenario set omits the real failure mode. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Keep scenario rationale, bounds, omitted risks, switch surfaces and the strongest conclusion shared across analyses. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
10. Govern overrides and irreversible actions
Define when accountable decision-makers may override the calculated rule and how reasons, conflicts and later review are recorded. Tighten evidence for irreversible action. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Would the same override be accepted if it favoured another stakeholder or outcome? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. A regulator overrides expected loss because a rare irreversible harm was not credibly quantified, documents the safeguard and commissions targeted evidence. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer governance to an adaptive experiment where stopping changes future learning. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. An override can be legitimate, but an undocumented override destroys auditability. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Record rule, override authority, reasons, dissent, mitigation, review date and the evidence that could reverse the choice. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
| Checkpoint | What to test |
|---|---|
| Decision question | Would the same override be accepted if it favoured another stakeholder or outcome? |
| Worked comparison | A regulator overrides expected loss because a rare irreversible harm was not credibly quantified, documents the safeguard and commissions targeted evidence. |
| Transfer test | Transfer governance to an adaptive experiment where stopping changes future learning. |
| Known boundary | An override can be legitimate, but an undocumented override destroys auditability. |
11. Answer the questions decision-makers ask
Explain why the frontier does not identify one winner, what a loss function adds, how regret differs from risk and when a decision can remain undecided. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Can the answer state both the mathematical result and the value judgement needed next? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. A manager asks for the optimal repair. The analyst shows the nondominated set, the selected loss rule and the switch point at which another repair wins. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the explanation to students using travel time, cost and safety as incompatible objectives. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. Do not use technical language to obscure who chose the tradeoff. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Publish plain-language questions, worked numbers, limits, appeal or review route and the current decision owner. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
Can one threshold settle every case?
No. A threshold belongs to a purpose, population, evidence base and consequence structure. Recheck it when any of those change.
Does more data always make the answer safer?
No. Repeated or poorly controlled data can reproduce the same blind spot and increase confidence without increasing validity.
When should an irreversible action pause?
Pause when a plausible unresolved condition could change the action and targeted evidence or a reversible alternative is proportionate to the risk.
What makes the process auditable?
Another person must be able to find the inputs, transformations, assumptions, alternatives, authority, decision and residual uncertainty without relying on memory.
12. Audit the choice and update it when evidence changes
Archive the frontier, loss specification, preferences, uncertainty, chosen action and observed outcomes. Reopen the choice when new data or objectives alter the tradeoff. In refuted or multi-criterion experimental models that leave several nondominated repairs or actions, the practical claim is which frontier point should guide a specified decision once outcomes, harms, thresholds and uncertainty are declared. The chapter therefore begins with the decision boundary: what the evidence must support, what it may suggest and what it cannot safely establish.
First-principles check. Could another analyst reproduce both the calculated ranking and the authorised judgement? Name the object being compared, the mechanism that could create error and the consequence of a wrong answer. Make the time window, population, method and authority explicit before accepting a score, label or workflow result.
Worked practice. Follow-up evidence reduces uncertainty and changes the regret frontier; the team reruns the decision rather than defending the historical choice. The important sequence is visible: observations come first, alternative explanations are tested, the bounded inference is stated and the action follows. This prevents the conclusion from becoming stronger merely because it is convenient.
Changed-context transfer. Transfer the audit to sequential evidence by versioning every frontier and decision rule. Carry over the reasoning steps, not the old threshold. Replace the local data, controls, costs and consequences, then ask whether the same conclusion still holds. A copied answer is not transfer if a hidden assumption changed.
Limit and misconception. A good earlier decision can become wrong under new evidence without having been negligent. The recurring danger is that the analyst selects a preferred compromise implicitly and presents it as if the mathematics chose it. A careful analysis marks the boundary and keeps competing explanations available until evidence genuinely excludes them.
Evidence record. Keep reproducible inputs, code, approvals, outcome monitoring, learning and verified specialist routes. Add source, date, operator or analyst, method and software version, exclusions and uncertainty. Preserve earlier states and link raw observations to derived decisions so that a later reviewer can challenge one step without losing the chain.
Decision-quality drill. Write one sentence naming the action supported now and one naming the strongest unsupported claim. Then change one plausible condition—threshold, method, missing record or stakeholder cost. If the action reverses, label the decision sensitive and set a review trigger.
Continue through verified routes
Return to the core Gold Standard framework or continue into the specialist routes that directly extend this decision. The external sources below define or test the concepts used here.
Core routes
Authoritative sources
- Li: Discordant relaxations of misspecified models
- Marmer and Otsu: Comparing misspecified models under a chosen measure of fit
- IJCAI: What Lies beyond the Pareto Front?
- Xu and Klabjan: Pareto regret analyses in multi-objective bandits
Guidance, methods and datasets can change. Recheck the linked source, local evidence and decision context before carrying the conclusion into a new case.