eduKateSG · Judgement · The Gold Standard of Education
Build a complete net-flow ranking without hiding the pairwise evidence or its option-set dependence
Choose the explanation you need, or follow all twelve chapters in order.
Full chapter index · Independent practice · Judgement subject guide
PROMETHEE methods turn criterion-by-criterion performance differences into pairwise preference degrees. Positive flow measures how strongly an option outranks others; negative flow measures how strongly others outrank it. PROMETHEE II subtracts the two to produce a complete ranking. That clarity is useful, but the net flow depends on thresholds, weights and which alternatives are present.
The examples below are invented. They connect raw differences to preference functions, flows and ranking, then show why a new near-duplicate option can reverse an existing order. PROMETHEE is treated as a decision aid whose assumptions stay visible, not an automatic truth machine.
Continue from The Gold Standard of Judgement within The Gold Standard of Education: learning that transfers into life and the How X Works learning library.
Full chapter index
See the mechanism · Chapters 1–2
Build the worked model · Chapters 3–5
Test what changes · Chapters 6–8
Check evidence and limits · Chapters 9–10
Practise and continue · Chapters 11–12
List feasible options as they existed at the decision date. Define every criterion’s direction, unit, population, horizon and evidence source. Remove unlawful or impossible options before ranking.
PROMETHEE compares a finite action set. Adding or removing options changes the average pairwise flows, so version the set and preserve excluded candidates with reasons.
Name the decision authority and stakeholders. Technical ranking cannot privately decide legitimate weights and thresholds.
Evidence check: for Define options, criteria and decision date, preserve the observation before the interpretation. Record population, binary action, outcome and time horizon. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
For each criterion, a preference function converts the difference between A and B into a degree from no preference to full preference. Common shapes include usual, U-shape, V-shape, level and linear functions.
Indifference and preference thresholds should reflect measurement resolution and meaningful difference, not be tuned to deliver a favourite winner. Some criteria may also need separate feasibility or veto logic outside a fully compensatory flow.
Record units and orientation before applying the function. Cost differences and benefit differences use opposite signs.
Evidence check: for Choose preference functions and thresholds, preserve the observation before the interpretation. Record whose tradeoff the threshold represents. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 3 OF 12 · Build the worked model
3. Compute pairwise unicriterion preferences
For ordered pair A,B, calculate A minus B in the beneficial direction on each criterion. Apply the chosen preference function and threshold. Repeat for B,A; the results need not be symmetric around one.
Keep the raw difference beside the transformed degree. A reader should see whether a preference came from a large measured gap or a threshold boundary.
Missing performance should not become zero preference silently. Use bounded scenarios, delay the comparison or collect data.
Evidence check: for Compute pairwise unicriterion preferences, preserve the observation before the interpretation. Record the formula, denominator and threshold weight. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Combine criterion preference degrees using declared normalised weights to obtain an overall preference index π(A,B). The positive flow for A averages π(A,x) across other options. The negative flow averages π(x,A). Net flow is positive minus negative.
In the fictional table, A has positive flow 0.62 and negative flow 0.28, giving net 0.34. PROMETHEE II ranks options by decreasing net flow.
Audit every pair and denominator. With n options, each flow averages over n minus one comparisons.
| Option | Positive flow φ+ | Negative flow φ− | Net flow φ | PROMETHEE II rank |
|---|---|---|---|---|
| A | 0.62 | 0.28 | 0.34 | 1 |
| B | 0.49 | 0.36 | 0.13 | 2 |
| C | 0.31 | 0.54 | −0.23 | 3 |
| D | 0.22 | 0.46 | −0.24 | 4 |
Evidence check: for Aggregate and calculate flows, preserve the observation before the interpretation. Record TP and FP counts recalculated at that threshold. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Weights determine how criterion-level preferences contribute to the pairwise index. Use a structured process with concrete cases and document whose judgements they represent.
Correlated criteria can count one consequence twice. Organise criteria hierarchically, test dependence and show alternative weight sets. Normalisation does not create legitimacy.
If stakeholders disagree, preserve separate scenarios rather than averaging away the conflict prematurely.
Check whether weights are being used as importance coefficients or as hidden exchange rates. In PROMETHEE they scale criterion-level preference degrees, which already depend on the chosen functions and thresholds. Doubling a weight does not have a standalone meaning without those transformations. Demonstrate the consequence with concrete A-versus-B cases near the thresholds. If participants reject the resulting comparisons, revise the model transparently and recheck earlier cases rather than changing a number only around the desired winner.
Document any criterion hierarchy used to derive the final weights. Rounding can alter close net flows, so compute with sufficient precision while displaying readable values and retaining the unrounded audit record.
Evidence check: for Elicit weights without double counting, preserve the observation before the interpretation. Record treat-all, treat-none and current-practice comparators. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
PROMETHEE I uses positive and negative flows to form a partial relation, allowing preference, indifference and incomparability. PROMETHEE II uses net flow to produce a complete order, even when positive and negative evidence tell different stories.
The complete ranking is easier to communicate but loses some structure. Report both flows beside the net score so a high positive and high negative option is not mistaken for an uncomplicated winner.
Choose the variant from the decision need, not the desire for one tidy list.
Evidence check: for Distinguish PROMETHEE I and PROMETHEE II, preserve the observation before the interpretation. Record the relevant range and uncertainty around differences. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 7 OF 12 · Test what changes
7. Inspect ties, close flows and stable relations
Rank gaps matter. Options separated by 0.001 should not be narrated like options separated by 0.3. Propagate measurement and threshold uncertainty to identify ties or unstable positions.
Examine pairwise edges that remain across plausible settings. A stable top group can be more defensible than one exact ordering.
Use visual profiles carefully; they summarise the model but do not replace the underlying evidence table.
Evidence check: for Inspect ties, close flows and stable relations, preserve the observation before the interpretation. Record calibration in the intended validation population. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 8 OF 12 · Test what changes
8. Test rank reversal when the option set changes
Because flows average comparisons against the current set, adding a new option can change existing scores. A near-copy of C may give some options extra favourable or unfavourable comparisons and reverse A versus B.
Rerun the original pair after adding, removing or splitting options. Ask whether the new option represents a real choice or duplicates an existing one. Preserve both versions.
Rank reversal is not automatically an error; it is a sensitivity that needs interpretation. It can be unacceptable when an irrelevant duplicate drives the decision.
Distinguish a genuinely new option from duplication. A new option may reveal information about the decision space or create a feasible compromise, making changed ranks reasonable. A near-identical copy added only through administrative splitting can distort the comparison set. Test clone alternatives by adding controlled copies with small perturbations, and report whether existing pairwise orderings survive. Also test deletion of clearly dominated or infeasible options after confirming that the method’s published ranking used the same set.
When the set is expected to evolve, define a reanalysis trigger instead of presenting the first order as permanent. Procurement shortlists, policy packages and technology choices often change between appraisal and commitment.
| Option set | A net flow | B net flow | A versus B | Audit question |
|---|---|---|---|---|
| A,B,C | 0.18 | 0.14 | A above B | Base set |
| A,B,C,C2 | 0.09 | 0.11 | B above A | Did C2 add real choice? |
| A,B,C,D | 0.16 | 0.13 | A above B | Different added option |
Evidence check: for Test rank reversal when the option set changes, preserve the observation before the interpretation. Record target prevalence and sampling weights. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 9 OF 12 · Check evidence and limits
9. Add measurement and scenario uncertainty
Replace each uncertain performance value with an interval or distribution supported by evidence. Recalculate preferences and ranks across draws or scenarios. Report rank acceptability or stable relations rather than one deterministic list when appropriate.
Threshold uncertainty can be as important as measurement uncertainty. Separate technical ranges from value disagreement because they require different next actions.
Do not assign probabilities to scenarios without a defensible basis.
Use robustness summaries that answer the decision question. A rank-frequency chart may show how often each option occupies each position, while a pairwise stability matrix shows how often A remains above B. The second can be clearer when exact middle ranks are volatile. Preserve the sampled inputs that produced every run so an unusual reversal can be reproduced. If weights represent several stakeholder perspectives, report results separately as well as together; a pooled distribution can hide systematic disagreement between groups.
Uncertainty analysis cannot repair an omitted criterion or an infeasible option. Reopen the model structure when residual concerns do not fit any current input range.
Evidence check: for Add measurement and scenario uncertainty, preserve the observation before the interpretation. Record resampling boundaries and external validation. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 10 OF 12 · Check evidence and limits
10. Know the limits and common misconceptions
PROMETHEE II’s complete order does not prove one option dominates. Compensation can occur through weights and netting of positive against negative flows. Hard constraints should be applied before ranking.
Results depend on preference functions, thresholds, weights, action set and data quality. A branded software output is not a substitute for those decisions.
Fairness and distributional harms can disappear inside aggregate criteria. Add subgroup evidence or non-compensable rules when justified.
The method does not decide whether a criterion belongs in the model, whether its measurement is ethically acceptable or whether the option set excludes affected communities. Those are prior governance questions. Keep a change log for criteria, functions and thresholds so later reviewers can separate data updates from value changes. A reproducible calculation with an illegitimate framing is still a poor decision aid.
Evidence check: for Know the limits and common misconceptions, preserve the observation before the interpretation. Record capacity, downstream harms and missing utilities. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 11 OF 12 · Practise and continue
11. Independent practice and changed-context transfer
Practice case: A leads B by 0.04 in net flow, but changing one preference threshold reduces the gap to 0.005. State how the recommendation and sensitivity note should change.
Changed context: add C2, a near-copy of C, and B moves above A. Recompute pairwise contributions that changed even though A and B data did not. Then remove a dominated option and test the order again.
Finish with the ranked set, model version, stable relations and the option-set change most likely to reverse the decision.
Evidence check: for Independent practice and changed-context transfer, preserve the observation before the interpretation. Record which changed context reverses the preferred rule. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
What is positive flow?
It is the average strength with which an option outranks the other options in the current set.
Why can PROMETHEE II reverse a rank?
Net flows depend on comparisons with all included options, so changing the set can change existing scores.
Is a complete ranking always preferable?
No. PROMETHEE I’s incomparability can be more honest when positive and negative evidence conflict.
Continue with outranking, concordance and veto thresholds and dominance, Pareto frontiers and non-compensable constraints.
Authoritative background includes the original PROMETHEE method paper, the original project-ranking article How to Select and How to Rank Projects, and recent peer-reviewed research on preference-function sensitivity and rank reversal. Sources and links were checked on 22 October 2026.
Evidence check: for FAQ, sources and connected reading, preserve the observation before the interpretation. Record the decision sentence that the curve is meant to support. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.