How Evidence Works | Transportability Boundary — When Does Evidence Stop Travelling Across People, Settings or Time?

A result works here.

Will it work there?

A transportability boundary is the point at which differences in people, setting, time, implementation, baseline state or surrounding system become large enough that evidence from one context no longer justifies the same confidence in another without additional support.

This is the fourth pillar beneath How Evidence Works. The master owns generalisation broadly. This page owns the travel question: where does valid evidence stop carrying the same weight?

Quick Read

Evidence does not automatically travel because the original study was strong. A result may depend on age, baseline capability, language, institutional support, implementation quality, dose, technology, incentives or historical conditions. Transportability therefore asks which features of the new case are causally or operationally relevant and whether they match the evidence-producing context. When similarity is high and the mechanism is stable, transfer can be reasonable. When important modifiers differ, confidence should shrink and local evidence should rise. Strong practice combines outside evidence with inside receipts rather than choosing between them.

source context → identify effect conditions → compare target context → preserve / reduce confidence → local implementation → local receipt → update transport boundary

Strong Internal Evidence Does Not Guarantee External Fit

A well-designed study can establish what happened under its conditions.

Transport asks whether those conditions resemble the new world enough for the result to travel.

Population Differences Matter

• age;
• baseline ability;
• language;
• prior knowledge;
• health/status;
• motivation;
• selection into the programme.

An average effect in one population may not estimate the same effect in another.

Setting Differences Matter

A method succeeds in a highly supported research environment.

Real classrooms may differ in class size, time, teacher expertise, resources and competing demands.

Implementation Is Part of the Evidence Route

Same programme name does not mean same programme exposure.

Transportability depends on whether the target system can reproduce the critical mechanism at sufficient quality.

Time Can Move the Boundary

Technology changes.

Curricula change.

Behaviour adapts.

Evidence that transported well ten years ago may need renewed validation now.

Mechanism Knowledge Helps Transport

If we understand why an effect occurs, we can ask whether the required mechanism exists in the target context.

Without mechanism, similarity can become superficial.

Conflict Often Reveals a Boundary

The third sibling, Evidence Conflict, owns disagreement among credible routes.

When results differ systematically by context, the conflict may be evidence that the original claim needs a transport condition.

Claim–Evidence Distance Grows When We Transport

The second sibling, Claim–Evidence Distance, owns general overreach.

Every move from observed context to new population, setting or time adds distance that must be justified.

Outside Evidence and Local Evidence Should Work Together

External research can provide a prior:

This intervention is plausible and has worked under related conditions.

Local evidence then asks:

Is it working here, for this learner or system, under this implementation?

Transportability Is Not “Every Case Is Unique”

If no evidence could ever travel, learning from others would be impossible.

The goal is calibrated transfer: identify relevant similarities and differences instead of assuming perfect universality or total uniqueness.

A Practical Transport Map

SOURCE EVIDENCE: Vocabulary method improves retention
SOURCE POPULATION: Secondary students
SOURCE SETTING: 20-minute daily practice, trained teachers
TARGET: Primary 5 learner
KEY DIFFERENCES:
- age
- vocabulary baseline
- session duration
KEY SIMILARITIES:
- retrieval mechanism
- spaced repetition
PLAN:
Use method at lower dose, measure local retention after 1 and 4 weeks

A 20-Lens Transportability Audit

1. What exact source claim is supported?
2. Who produced the source evidence?
3. What source population was studied?
4. What target population is different?
5. What baseline capability differs?
6. What setting differs?
7. What implementation differs?
8. What dose differs?
9. What incentives differ?
10. What technology differs?
11. What time period differs?
12. What mechanism is believed to carry the effect?
13. Does that mechanism exist in the target?
14. Which modifiers may change effect size?
15. What evidence conflict already suggests boundaries?
16. How much should confidence shrink?
17. What can be safely tried locally?
18. What local receipt should be measured?
19. What result would widen the boundary?
20. Does the final claim distinguish “worked there” from “works here” honestly?

Final Thought: Evidence Travels on Conditions

The right question is not whether evidence generalises everywhere. It is which parts of the evidence-producing world must remain true for the result to keep travelling.