eduKateSG · Judgement · The Gold Standard of Education
Use the Best-Worst Method to make preference ratios inspectable before they become weights
Choose the explanation you need, or follow all twelve chapters in order.
Full chapter index · Independent practice · Judgement subject guide
The Best-Worst Method, or BWM, is a multi-criteria weighting method. A decision-maker identifies the most important and least important criteria, compares the best criterion with every criterion, and compares every criterion with the worst. An optimisation model finds weights that fit those ratios as closely as possible.
The method can reduce the number of pairwise comparisons, but fewer comparisons do not remove judgement. This guide builds the original and linear models, works a transparent four-criterion example, checks consistency and sensitivity, and separates criterion weights from alternative performance. It also shows when disagreement, thresholds or non-compensable constraints require a different model.
Continue from The Gold Standard of Judgement within The Gold Standard of Education: learning that transfers into life and the How X Works learning library.
Full chapter index
See the mechanism · Chapters 1–2
Build the worked model · Chapters 3–5
Test what changes · Chapters 6–8
Check evidence and limits · Chapters 9–10
Practise and continue · Chapters 11–12
CHAPTER 1 OF 12 · See the mechanism
1. Define the decision before choosing best and worst
Begin with the decision owner, alternatives, criteria, affected people and time horizon. BWM weights criteria; it does not define the decision for you. If the criteria omit a serious consequence, mathematically consistent weights cannot restore it.
Write each criterion so that its direction and measurement are clear. “Access” could mean travel time, physical accessibility, service coverage or affordability. Combining these into one vague word makes later ratios impossible to interpret.
Apply hard constraints before trade-off weighting. An alternative that violates law, safety or a required capability should not become acceptable merely because it performs well elsewhere.
State whether one person, a facilitated group or several separate participants will provide preferences. A group average can hide disagreement. Preserve individual inputs before aggregation.
Evidence check: for Define the decision before choosing best and worst, preserve the observation before the interpretation. Record decision owner, alternatives, criteria and feasibility screen. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Suppose there are n criteria. The decision-maker selects a best criterion B and a worst criterion W for the current decision. “Best” means most important in the weighting task; “worst” means least important. It does not mean morally good or bad.
The best-to-others vector records how strongly B is preferred to each criterion j: a_Bj. The others-to-worst vector records how strongly each criterion j is preferred to W: a_jW. The self-comparisons a_BB and a_WW equal one.
Use a declared ratio scale, commonly integers from one to nine in the original method. A value of four means the decision-maker intends a four-to-one importance ratio, not merely “somewhat more important.” That strong numerical meaning should be explained before elicitation.
Ask for reasons as well as numbers. A short note beside each comparison helps later review and reveals when two people interpret a criterion differently.
Evidence check: for Understand the two comparison vectors, preserve the observation before the interpretation. Record best-to-others and others-to-worst vectors. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
If comparisons were perfectly consistent, the implied ratio between the best and worst would agree through every intermediate criterion. Algebraically, a_Bj multiplied by a_jW should equal a_BW for each j.
For four criteria, suppose Safety is best and Cost is worst. If Safety is four times Reliability and Reliability is twice Cost, the implied Safety-to-Cost ratio is eight. If the direct judgement says Safety is only three times Cost, the set is inconsistent.
Inconsistency is not dishonesty. It can reveal imprecise thinking, fatigue, scale compression or a genuine difficulty in expressing complex priorities as fixed ratios.
Calculate the implied products before solving. This quick diagnostic shows which comparisons create tension and gives the decision-maker a chance to clarify definitions rather than mechanically forcing a numerical fit.
Evidence check: for Derive the consistency relationship, preserve the observation before the interpretation. Record ratio meaning and multiplicative consistency. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 4 OF 12 · Build the worked model
4. Work a perfectly consistent fictional example
A fictional school chooses weights for comparing three feasible learning-support plans. The criteria are Safety, Access, Reliability and Cost. The decision owner identifies Safety as best and Cost as worst.
The best-to-others vector for Safety, Access, Reliability and Cost is [1, 2, 4, 8]. The others-to-worst vector is [8, 4, 2, 1]. Every intermediate product equals eight, so the judgements are perfectly multiplicatively consistent.
Weights proportional to [8, 4, 2, 1] reproduce every ratio. Normalising by the sum, 15, gives approximately 0.533 for Safety, 0.267 for Access, 0.133 for Reliability and 0.067 for Cost.
These weights belong to the stated criteria and decision. They do not mean safety is universally eight times cost. The table makes the ratios, normalisation and context visible.
| Criterion | Best-to-others | Others-to-worst | Normalised weight |
|---|---|---|---|
| Safety | 1 | 8 | 0.533 |
| Access | 2 | 4 | 0.267 |
| Reliability | 4 | 2 | 0.133 |
| Cost | 8 | 1 | 0.067 |
Evidence check: for Work a perfectly consistent fictional example, preserve the observation before the interpretation. Record every fictional ratio, product and weight. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
The original BWM seeks non-negative weights summing to one while minimising the largest absolute departure between weight ratios and stated ratios. It controls deviations in w_B / w_j versus a_Bj and w_j / w_W versus a_jW.
The linear formulation replaces ratio deviations with constraints such as |w_B − a_Bj w_j| ≤ ξ and |w_j − a_jW w_W| ≤ ξ. The objective is to minimise ξ, subject to the weights summing to one and remaining non-negative.
The optimal ξ records the largest fitted deviation in this linear model. A small value means the weights closely reproduce the stated comparisons. It does not prove that the criteria are complete or the decision is wise.
Preserve the exact model version. Nonlinear, linear, multiplicative and Bayesian extensions do not have identical properties or consistency measures. A label saying only “BWM” is insufficient for reproducibility.
Evidence check: for Formulate the optimisation problem, preserve the observation before the interpretation. Record linear model, objective and weight constraints. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
The original BWM introduced a consistency ratio using the achieved inconsistency and a consistency index linked to the direct best-to-worst comparison. Later work proposed alternative input-based and output-based measures.
Do not mix a threshold from one formulation with a statistic from another. State the formula, scale, model version and reference used. A reported ratio without those details cannot be checked.
Treat consistency as a diagnostic, not a pass–fail stamp. If inconsistency is high, identify which comparisons drive it. Ask whether the decision-maker wants to revise a judgement after seeing the implication. Never delete an inconvenient comparison solely to improve the score.
Even perfect consistency can encode a poor model. The fictional [1,2,4,8] example is internally exact, yet it would still fail if “Safety” combined unrelated harms or if an affected group had no voice.
Evidence check: for Read consistency ratios carefully, preserve the observation before the interpretation. Record consistency formula tied to model version. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 7 OF 12 · Test what changes
7. Move from criteria weights to alternative scores
BWM produces weights. Alternatives still need performance values on each criterion. Define scales, measurement periods, uncertainty and whether higher or lower is better.
Normalisation matters. Converting dollars, minutes and probabilities into comparable scores can change the final ranking. Preserve raw performance data beside normalised values so that readers can see what a score means.
An additive weighted sum assumes that improvement on one criterion can compensate for weakness on another at the stated rates. That may be inappropriate for minimum safety, rights or threshold requirements.
Calculate contributions criterion by criterion. If Plan A wins because its cost advantage overwhelms a small safety difference, the decision owner should be able to see that mechanism rather than only the final total.
Evidence check: for Move from criteria weights to alternative scores, preserve the observation before the interpretation. Record raw performance, normalisation and contributions. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 8 OF 12 · Test what changes
8. Handle multiple decision-makers without hiding disagreement
One route is to elicit a separate BWM model from each participant, inspect consistency, then compare resulting weights. Another is to deliberate toward shared best, worst and ratios. These are different social processes.
Geometric aggregation can preserve ratio structure better than a simple arithmetic average in some designs, but aggregation always makes a normative choice about whose judgement counts and how.
Show distributions or ranges. If one group gives Access a weight near 0.40 and another near 0.10, a mean near 0.25 is not evidence of consensus. Investigate whether their circumstances or criterion definitions differ.
Protect participation. Technical language, status and meeting format can shape elicited ratios. A model should make disagreement legible, not pressure people into false numerical unity.
Evidence check: for Handle multiple decision-makers without hiding disagreement, preserve the observation before the interpretation. Record individual inputs, participation and aggregation. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 9 OF 12 · Check evidence and limits
9. Test sensitivity, alternative models and rank stability
Vary each judgement across a plausible range and re-solve. Record which ratio changes the leading alternative and how far it must move. A result that flips after a small change deserves a different decision posture from one that remains stable across broad ranges.
Compare model variants where appropriate. The linear and original nonlinear forms can produce different weights when inputs are inconsistent. Explain the reason before selecting one.
Add or remove an alternative and check whether rankings of existing options change through the performance-normalisation step. Criterion weights may be stable while alternative scores are not.
Use a holdout question: ask the decision-maker for one additional ratio not used in fitting, then compare it with the implied weight ratio. This does not validate the whole model, but it tests whether the fitted weights predict another judgement.
| Test | Change | Question | Warning exposed |
|---|---|---|---|
| Ratio range | Vary one comparison | Where does the leader switch? | False precision |
| Model variant | Linear vs nonlinear | Are weights formulation-dependent? | Method ambiguity |
| Holdout ratio | Predict unused judgement | Do fitted weights generalise? | Overfit elicitation |
| Performance scale | Change normalisation | Do alternative ranks move? | Score construction |
Evidence check: for Test sensitivity, alternative models and rank stability, preserve the observation before the interpretation. Record sensitivity, holdout and rank stability. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 10 OF 12 · Check evidence and limits
10. State boundaries and common misconceptions
BWM is not best-worst scaling used in survey experiments, although the names sound similar. BWM derives criterion weights from two structured comparison vectors. Best-worst scaling presents choice sets and models repeated selections.
Selecting one best and worst criterion does not mean intermediate criteria are ignored. They are compared through both vectors. Nor does fewer comparisons guarantee less bias; anchoring on the chosen best and worst can influence later numbers.
Weights are not probabilities, utilities or observed frequencies unless a separate model establishes that meaning. A 0.50 weight does not say the criterion has a fifty-percent chance of occurring.
Do not use BWM to bypass expertise, legal duties or stakeholder rights. It organises expressed trade-offs under assumptions. The final judgement still belongs to the authorised decision process.
Evidence check: for State boundaries and common misconceptions, preserve the observation before the interpretation. Record method identity, probability and authority limits. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Case A uses the consistent vectors [1,2,4,8] and [8,4,2,1], producing weights proportional to [8,4,2,1]. Safety clearly dominates and the fit has no inconsistency under the teaching model.
Case B changes one judgement: Access is said to be eight times Cost instead of four, while Safety remains twice Access and eight times Cost. The implied path Safety-to-Access-to-Cost is sixteen, conflicting with the direct ratio eight. The optimiser must compromise.
Before accepting the compromise, ask whether Access changed meaning. Suppose the decision now concerns a learner with mobility barriers, so accessible travel became an essential condition rather than a trade-off criterion. The correct response may be to move Access into the feasibility screen, not merely increase its weight.
The transfer lesson is that inconsistency can reveal a changed decision structure. Repair the model when the context changes instead of polishing the ratio.
Evidence check: for Run a changed-context transfer test, preserve the observation before the interpretation. Record the newly essential Access changed case. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Archive the decision statement, alternatives, feasibility screen, criterion definitions, participants, elicitation instructions, best and worst selections, both comparison vectors, reasons, model formulation, solver, optimal weights, ξ, consistency formula, revisions, raw performance data, normalisation, contribution table, sensitivity ranges and final decision record.
Common questions: Does BWM choose the best alternative automatically? No. Does a low consistency ratio validate the criteria? No. Can groups use it? Yes, but disagreement and aggregation must be explicit. Are its weights probabilities? No. Is it the same as best-worst scaling? No.
Continue with AHP consistency and rank reversal, swing weighting and reference levels, and dominance and non-compensable constraints. Read Rezaei’s original 2015 BWM paper, the 2016 linear model, and the author’s paper and worked-material index.
Evidence check: for Keep the evidence route and continue, preserve the observation before the interpretation. Record the narrowest defensible weighting claim. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.