eduKateSG · Judgement · The Gold Standard of Education
Use DEMATEL to organise perceived influence without mistaking it for experimental causation
Choose the explanation you need, or follow all twelve chapters in order.
Full chapter index · Independent practice · Judgement subject guide
The Decision-Making Trial and Evaluation Laboratory method, usually called DEMATEL, turns judged directional influences among factors into a matrix and a network map. It can reveal which factors are perceived as central and which sit on the net-influencing or net-receiving side of the model.
Its usefulness depends on a careful boundary. The entries normally come from expert judgement, not controlled interventions. A DEMATEL arrow is therefore an elicited influence within a defined system, not automatic proof that changing one factor will cause the predicted outcome.
Continue from The Gold Standard of Judgement within The Gold Standard of Education: learning that transfers into life and the How X Works learning library.
Full chapter index
See the mechanism · Chapters 1–2
Build the worked model · Chapters 3–5
Test what changes · Chapters 6–8
Check evidence and limits · Chapters 9–10
Practise and continue · Chapters 11–12
State who will use the result, what decision it informs and which factors are inside the model. A map for diagnosing delayed school interventions is not the same as a map for allocating budgets or predicting student outcomes.
Choose factors at a comparable level. Mixing broad outcomes such as “learning quality” with narrow actions such as “send a reminder” creates ambiguous comparisons. Define each factor with inclusion and exclusion examples.
Set the time horizon and direction convention. “Factor i influences factor j” must have the same meaning for every respondent. Specify whether influence means immediate effect, total effect, frequency, severity or perceived leverage.
Record what lies outside the boundary. Regulation, resources and population differences may shape every judgement even when they are not nodes. The map is conditional on those background conditions.
Evidence check: for Define the decision and system boundary, preserve the observation before the interpretation. Record decision owner, system boundary and factor dictionary. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Create an n by n matrix for n factors. Entry a(i,j) records the judged direct influence of factor i on factor j. The diagonal is commonly zero because a factor is not asked to influence itself directly.
Use a short anchored scale, for example zero for no direct influence through four for very high direct influence. Give respondents worked examples and practise distinguishing direct from indirect paths. If A affects C only through B, the direct A-to-C score should not silently include the whole chain.
Collect individual matrices before aggregation. This preserves disagreement and prevents a dominant voice from shaping a group matrix invisibly. Record expertise, conflicts of interest and the perspective represented.
Missing or “cannot judge” is not the same as zero. Decide how uncertainty will be handled rather than forcing every blank into no influence.
Evidence check: for Design the direct-influence elicitation, preserve the observation before the interpretation. Record direction convention and influence scale. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
An arithmetic mean is common, but it assumes the scale and respondents justify that operation. Median, trimmed mean or subgroup analysis may be more robust when judgements are skewed or expertise differs.
Do not weight experts merely by seniority. If weights are used, define them prospectively from relevant experience, task knowledge or calibration. Show both weighted and unweighted results.
Calculate dispersion for every directed pair. Two opposite subgroups can average to a moderate influence that nobody actually believes. A disagreement map can be as useful as the consensus map.
Return summaries to participants for a structured check, but do not edit responses until the map looks tidy. Document any second round and whether it replaced or supplemented the first.
Evidence check: for Aggregate while preserving disagreement, preserve the observation before the interpretation. Record individual matrices and disagreement. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Imagine a learning-support team examining four factors: diagnosis quality D, response speed S, family access A and follow-through F. After averaging independent judgements, the direct-influence scores use a zero-to-four scale.
The team judges D strongly influences S and F, S moderately influences F, A influences S and F, and F has smaller feedback effects. The maximum row sum is eight, so the matrix is normalised by dividing every entry by eight. Normalisation keeps the matrix within the range needed for the total-relation calculation.
Let the normalised direct matrix be N. The total-relation matrix is T = N(I − N)⁻¹, equivalently N + N² + N³ and so on when the series converges. T contains the modelled direct and indirect influence totals.
For teaching, calculate one second-order path: if D-to-S is 0.50 and S-to-F is 0.25, the D-to-S-to-F contribution in N² is 0.125. This shows why total influence can exceed the direct entry without adding a new judgement.
| Factor | Illustrative direct role | Model question | Caution |
|---|---|---|---|
| Diagnosis quality D | Influences speed and follow-through | What is dispatched? | Judged, not experimental |
| Response speed S | Influences follow-through | What paths continue? | Time horizon matters |
| Family access A | Influences speed and follow-through | Whose perspective? | Panel representation |
| Follow-through F | Receives and feeds back | What is received? | Feedback is modelled |
Evidence check: for Work a fictional four-factor matrix, preserve the observation before the interpretation. Record every fictional matrix step and path. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 5 OF 12 · Build the worked model
5. Interpret row sums, column sums and position
For factor i, let Dᵢ be the row sum of T: total influence dispatched to other factors. Let Rᵢ be the column sum: total influence received. Some sources use r and c or other symbols; define the notation.
Dᵢ + Rᵢ is prominence or involvement. A large value means the factor is deeply connected in the elicited system, either as sender, receiver or both. It does not by itself mean the factor should receive the most resources.
Dᵢ − Rᵢ is net relation. A positive value places a factor on the net-influencing side; a negative value places it on the net-receiving side. This is a balance of modelled flows, not a probability of being a root cause.
Plot prominence horizontally and net relation vertically. Keep numerical labels and uncertainty beside the map so a visually dramatic position is not mistaken for precision.
Evidence check: for Interpret row sums, column sums and position, preserve the observation before the interpretation. Record normalisation and total-relation formula. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 6 OF 12 · Test what changes
6. Choose thresholds without manufacturing a story
The total-relation matrix is dense because indirect paths connect many factors. A threshold can hide small edges so the diagram remains readable. Common choices include a mean, percentile or stakeholder-selected level.
The threshold changes the displayed map, not the underlying T matrix. Report both. A factor can retain high prominence even when many of its individual edges fall below the display threshold.
Run several plausible thresholds and note which nodes, directions and components persist. If the “key causal structure” disappears after a small threshold change, the display is fragile.
Do not choose the cutoff after seeing which story looks most persuasive. Predefine a rule or present a threshold path showing how the map simplifies.
Evidence check: for Choose thresholds without manufacturing a story, preserve the observation before the interpretation. Record D plus R and D minus R meanings. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 7 OF 12 · Test what changes
7. Distinguish a perceived influence map from causal inference
DEMATEL encodes informed beliefs about influence. It can organise mechanisms and generate intervention hypotheses. It does not estimate counterfactual effects from observed or experimental data unless a separate design supplies that evidence.
Feedback cycles are allowed in the matrix. That is useful for systems thinking but differs from many statistical causal diagrams, which impose acyclicity. The word “cause group” in DEMATEL literature should be read as net-influencing group within the model.
Validate important arrows against process data, longitudinal evidence, experiments or natural experiments where feasible. An arrow from diagnosis quality to response speed can be tested by measuring whether improving diagnostic turnaround changes response time.
If the result is used for action, state the bridge: “Experts identify this as a plausible leverage point; a pilot will test the effect.” This is stronger than either ignoring expertise or treating it as proof.
Evidence check: for Distinguish a perceived influence map from causal inference, preserve the observation before the interpretation. Record thresholded display versus full matrix. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Repeat calculations with alternative aggregation rules, expert weights and subgroups. Examine whether factor rankings and net-relation signs remain stable. A sign near zero should not be labelled decisively causal or affected.
Test scale interpretation. Some respondents may use four only for rare extreme influence while others use it often. Training, anchoring vignettes or within-person normalisation can reduce but not eliminate this difference.
Compare classical crisp scoring with a fuzzy variant only when linguistic uncertainty justifies the added machinery. A more complex method is not automatically more accurate.
Bootstrap respondents to obtain empirical ranges for prominence and relation. With a small expert panel, report that the intervals describe panel variability, not population-level statistical certainty.
Evidence check: for Test scale, respondent and model sensitivity, preserve the observation before the interpretation. Record respondent and model sensitivity. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 9 OF 12 · Check evidence and limits
9. Connect the map to alternatives and constraints
A high net-influence factor is not automatically the best intervention. Add feasibility, cost, ethics, delay, reversibility and evidence strength. DEMATEL maps structure; it does not complete the decision.
Avoid double-counting if DEMATEL-derived prominence becomes a weight in another multicriteria method. Explain whether the second method is using judged interdependence, stakeholder values or both.
Screen non-negotiable constraints before compensatory scoring. An option that violates safety or rights should not return through a high score elsewhere.
For each proposed action, write the assumed path through the map and a measurable near-term indicator. This turns a diagram into a testable implementation plan.
| Layer | Question | Evidence needed | Failure avoided |
|---|---|---|---|
| Influence map | What is perceived to affect what? | Expert matrices | Unstructured intuition |
| Feasibility | Can the action be delivered? | Cost and capacity | Impossible recommendation |
| Causal test | Does changing it alter outcomes? | Pilot or longitudinal design | Causal overclaim |
| Rights screen | What must not be traded away? | Policy and ethics | Compensatory harm |
Evidence check: for Connect the map to alternatives and constraints, preserve the observation before the interpretation. Record feasibility, causal testing and rights. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 10 OF 12 · Check evidence and limits
10. Report participation, limits and reproducibility
List factors, definitions, respondent roles, recruitment, elicitation instructions, scale anchors, missing-data rule, aggregation, normalisation, matrix formula, threshold and software. Publish the direct and total matrices when confidentiality permits.
Preserve minority views when the decision affects people not represented in the panel. Add student, family or frontline perspectives rather than asking technical experts to speak for every value.
Common computational errors include transposing the matrix, swapping D and R, using inconsistent normalisation, rounding too early and interpreting a thresholded diagram as the full matrix. An independent recalculation is inexpensive.
State the method’s limits beside the conclusions. The factor set may be incomplete, influence is context-dependent, and respondent judgement can reflect institutional habits as well as real mechanisms.
Evidence check: for Report participation, limits and reproducibility, preserve the observation before the interpretation. Record participation and computational limits. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Case A produces high prominence and positive net relation for diagnosis quality. The team treats it as a candidate leverage point and designs a pilot, not as proven causation.
Case B adds a missing factor: staff capacity. Diagnosis quality’s net relation falls because capacity receives and transmits much of the earlier influence. The first map was not wrong mathematically; its boundary was incomplete.
Case C separates school leaders and frontline staff. Their mean maps look similar, but the access-to-speed edge differs sharply. A subgroup result prevents a false consensus.
Case D raises the display threshold. The access factor disappears from the picture although its total prominence remains material. The correct interpretation is that its individual edges are diffuse, not that access ceased to matter. Transfer means carrying definitions, matrices and sensitivity—not just the final diagram.
Evidence check: for Run a changed-context transfer test, preserve the observation before the interpretation. Record the missing-capacity changed context. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Archive the decision question, factor dictionary, system boundary, respondent selection, consent, individual matrices, aggregation, normalisation constant, total-relation calculation, D and R values, thresholds, sensitivity runs, subgroup results, external validation, implementation plan and exact claims.
Common questions: Does DEMATEL discover objective causes? No. Can it include feedback? Yes. Is D + R a priority weight? Not automatically. Does positive D − R prove a root cause? No. Can a threshold change the story? Yes. Should respondent disagreement be averaged away? No.
Continue with AHP consistency and rank reversal, Best-Worst Method preference ratios, and PROMETHEE outranking flows. Read the open worked DEMATEL matrix procedure, the systematic methodological review, and the open study of threshold sensitivity.
Evidence check: for Keep the evidence route and continue, preserve the observation before the interpretation. Record the narrowest defensible influence claim. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.