An explanation can sound elegant and still be wrong.
This is a particular danger with “How X Works” writing. Mechanism stories are satisfying. They connect causes to effects, turn complexity into sequence and give the reader a sense that the world has become legible.
That satisfaction is not evidence.
For eduKateSG’s How X Works library, explanation testing is the discipline that asks whether a proposed mechanism deserves confidence, where its limits are and what would change our mind.
Start by Separating Claim From Story
A story can contain many claims at once.
“The queue grew because demand rose, which overloaded capacity, delayed service and caused more arrivals to overlap.” That sentence contains claims about demand, capacity, timing and causation.
Testing begins by separating them.
- Did demand actually rise?
- Was capacity fixed?
- Did utilisation approach a critical level?
- Did waiting time rise after the demand change?
- Were there competing causes such as a service slowdown?
Once the claims are visible, evidence can be matched to them.
Evidence Must Sit Close Enough to the Claim
Evidence can support less than we think.
An observed association does not automatically establish the mechanism that produced it. A single successful case does not prove general reliability. A policy document tells us what was intended, not necessarily what happened. A sensor tells us what it measured, not everything about the system.
This is why the wider eduKateSG evidence work distinguishes the distance between a claim and the observations underneath it. The longer the inferential journey, the more explicit the assumptions should become.
Test the Sequence
A mechanism has order.
If A is said to cause B through C, then the timing should be compatible with that sequence. The relevant state of A should exist before B, and C should occur in a way consistent with the proposed path.
This sounds obvious, yet many weak explanations are built from events that merely occurred near one another.
Test the Necessary Links
Ask what has to be present for the mechanism to work.
If a payment allegedly moved through a particular rail, there should be records consistent with that path. If a learning intervention allegedly worked through retrieval, the learner must actually have attempted retrieval rather than simply rereading. If a control system acts through feedback, there must be a measurement returned to the controller.
Missing necessary links weaken the explanation even if the final outcome occurred.
Look for Alternative Mechanisms
A strong explanation does not win because it is imaginable. It wins because it explains the evidence better than plausible alternatives.
Suppose a student improves after tuition. The mechanism might involve clearer explanation, more practice, better feedback, increased study time, greater confidence, maturation or a combination of these. A confident claim should not quietly attribute every improvement to the favourite mechanism.
Likewise, a city policy followed by an improved outcome does not by itself prove the policy caused the change. The world kept moving while the policy was introduced.
Run the Mechanism Forward
Forward testing asks: if the mechanism is true and these inputs are present, what should we expect to observe next?
This converts explanation into prediction.
If the proposed problem is a capacity bottleneck, adding capacity at the correct point should change the queue pattern. If the problem is missing prerequisite knowledge, repairing that prerequisite should improve several downstream tasks, not only the exact question practised.
Prediction is powerful because it gives the explanation a chance to fail.
Run It Backwards
Backward testing begins from the outcome and asks what must have been true upstream.
If a shipment was delivered correctly, identity, routing and custody had to survive enough of the journey. If a learner solved an unfamiliar problem independently, some usable representation and retrieval path had to exist before the final answer.
If the required upstream evidence is absent, the explanation needs revision or greater uncertainty.
Rotate the Receiver
Some explanations work only because they adopt the viewpoint of the system designer.
Rotate to the receiver.
A public service may be “available” in an administrative sense while inaccessible to a person who cannot use the required interface. A lesson may be “covered” because the teacher explained it while the learner remains unable to retrieve it. A parcel may be “delivered” according to a scan while the customer cannot find it.
The receiver test catches explanations that confuse process completion with outcome achievement.
Probe the Boundary Conditions
Every useful mechanism has conditions under which it stops working, changes form or becomes dominated by another effect.
- At what scale does the mechanism change?
- Under what load does it saturate?
- Which population or setting was the evidence drawn from?
- Does the mechanism depend on a particular technology, law or institution?
- What happens when a key assumption is removed?
Knowing the boundary is part of knowing the mechanism. This aligns with eduKateSG’s How Knowledge Works | Boundary Recognition: intellectual strength includes knowing when a model stops applying.
Use Counterfactual Questions Carefully
One useful test is to ask what would have happened if a proposed cause were absent.
Would the queue still have grown? Would the student still have improved? Would the structure still have failed? Would the institution still have produced the outcome?
We cannot always observe the counterfactual directly. But the question exposes what causal claim the explanation is really making.
Test Failure Predictions Too
A mechanism should tell us not only how success occurs, but how failure should appear.
If the mechanism depends on a particular handoff, then breaking that handoff should produce a characteristic downstream signature. If the mechanism depends on feedback, delayed or corrupted feedback should change behaviour in a predictable way.
This is one reason failure mapping is such a strong companion to explanation testing.
Distinguish Confidence From Certainty
Mechanism explanations often combine well-established parts with uncertain links.
A responsible article should be able to say which is which.
- Well established
- strongly supported
- plausible but incomplete
- contested
- unknown
Precision is not weakened by calibrated uncertainty. It is strengthened by it.
The Warehouse Evidence Discipline
Before combining sources, classify what each one actually provides.
Is it an observation, measurement, policy, model, expert interpretation, historical artifact, primary record or secondary summary? Does it independently support the claim, or is it repeating the same upstream source?
A stack of dependent citations is not the same as several independent lines of evidence.
A Compact Explanation Test
- State the claim precisely.
- Identify the proposed mechanism.
- Match evidence to each important link.
- Check temporal order.
- Consider alternatives.
- Run the mechanism forward.
- Run it backward.
- Rotate the receiver.
- Probe boundary conditions.
- State confidence and remaining unknowns.
A mechanism deserves belief not because it is a good story, but because it survives contact with evidence, alternatives, boundaries and prediction.
Why This Matters for How X Works
The internet has no shortage of fluent explanations. The valuable work is to build explanations that know what they claim, show what connects the steps and respect where knowledge ends.
That is the standard the How X Works library should keep raising as it expands across science, mathematics, language, infrastructure, finance, institutions and civilisation.