HOW INTELLIGENCE WORKS · HYPOTHESIS TESTING · eduKateSG
How Intelligence Chooses Evidence That Can Separate Competing Explanations
Hypothesis testing is the intelligence process that chooses observations, comparisons or interventions capable of changing confidence among competing explanations. The strongest test is not the one that gathers the most data, but the one that creates the clearest separation.
Candidate explanations → predicted differences → choose discriminator → observe → weight evidence → update confidence → reject, retain or refine.
This article belongs to the How Intelligence Works series. The canonical owner of scientific research remains How Scientific Research Works. This pillar isolates the cognitive job of choosing discriminating tests across science, learning, diagnosis and everyday reasoning.
The Discrimination Problem
If two hypotheses both predict the evidence you already have, collecting more of the same may add confidence without resolving the real uncertainty.
Testing asks: what result would one explanation expect more strongly than another?
A good test is designed around where the hypotheses disagree.
1. Testing Begins With Explicit Alternatives
A test becomes more informative when the competing explanations are stated clearly enough that each makes different predictions.
Hypothesis Generation owns candidate creation. Testing begins after enough alternatives exist to make evidence comparative rather than decorative.
2. Predictions Turn Explanations Into Tests
A hypothesis earns testability when it implies something observable: a trend, absence, timing difference, response to intervention or relationship among variables.
The more specific the prediction, the more informative a mismatch can become.
Vague hypotheses survive too easily because almost any result can be reinterpreted as compatible.
3. The Best Test Has High Information Value
A test has high information value when different outcomes would move belief meaningfully. Repeating a measurement already almost guaranteed under every hypothesis may be precise but uninformative.
| Test feature | Why it matters |
|---|---|
| Discriminating | Hypotheses predict different outcomes |
| Reliable | Noise is low enough to interpret |
| Relevant | The observation bears on the actual claim |
| Feasible | The test can be run within constraints |
| Ethical | The evidence is obtained without unacceptable harm |
| Timely | The result arrives before the decision expires |
4. Testing Is Not Confirmation Hunting
Weak testing asks for evidence that supports the preferred explanation. Strong testing asks what result would make the preferred explanation less credible.
This asymmetry matters because confirming examples are often easy to find when hypotheses are broad.
Evidence becomes powerful when the preferred hypothesis takes a real risk.
5. Hypothesis Testing in Mathematics
In mathematics, examples can test conjectures provisionally and counterexamples can destroy universal claims immediately.
Formal proof remains the canonical route for establishing many mathematical propositions. How Mathematical Proof Works owns that terrain.
Hypothesis testing helps before proof: choose examples near boundaries, test degenerate cases and search for structures most likely to expose a hidden exception.
6. Hypothesis Testing in Science
Scientific tests may use experiments, natural comparisons, measurement, replication or model predictions.
Interventions are especially powerful for causal questions because they can separate mere association from mechanism when design and assumptions permit.
The companion article Causal Reasoning owns the interpretation of causal structure.
7. Hypothesis Testing in Learning Diagnosis
If a learner’s error might come from vocabulary or mathematics, a good diagnostic task changes one factor while holding others stable.
A teacher can simplify language while preserving mathematical structure, or provide vocabulary support while changing the quantity relationship.
The point is not more questions. It is questions designed to separate plausible causes.
8. Negative Results Are Information
A well-designed test that fails to support the preferred hypothesis can improve the map more than a weak positive result.
Negative results narrow the possibility space, reveal missing mechanisms and prevent wasted search—provided the test itself had enough sensitivity and relevance.
Testing therefore evaluates both the hypothesis and the instrument used to challenge it.
9. Hypothesis-Testing Failure Atlas
| Failure | What happens | Repair |
|---|---|---|
| Confirmation test | The test can only make the favourite look good | Ask what would count against it |
| Non-discriminating evidence | All hypotheses predict the same result | Find a separating condition |
| Measurement blindness | Noise is mistaken for theory failure | Inspect instrument quality |
| Scope mismatch | The test occurs outside the hypothesis’s operating conditions | Align domain and claim |
| Outcome fishing | Only favourable results are emphasised | Predefine key outcomes |
| Premature rejection | One weak result eliminates a robust model | Weight evidence proportionately |
| Endless testing | No stopping rule exists | Define decision threshold |
10. Hypothesis Testing and Evidence Weighting Are Different
Testing chooses what evidence to obtain. Evidence weighting decides how much the obtained evidence should alter belief.
The companion article Evidence Weighting owns reliability, independence, relevance and diagnosticity after evidence arrives.
11. Teams Need Discriminator Ownership
Teams can argue for hours when nobody owns the question, “What observation would actually settle this?”
A strong group names the competing models, chooses a discriminating test, assigns the evidence owner and agrees in advance how different results will update the decision.
The best meeting sometimes ends not with agreement, but with a test that makes tomorrow’s disagreement smaller.
12. Institutions Need Testable Policies
Policies often embody hypotheses about behaviour, incentives and outcomes. If the expected return is never specified, institutional learning becomes difficult.
Strong institutions define what evidence would show the policy is working, failing or producing unacceptable side effects.
This makes policy corrigible rather than ideological.
13. Artificial Intelligence and Hypothesis Testing
AI systems can propose tests, compare predictions and help search large evidence spaces. They can also generate post-hoc explanations that make almost any result appear compatible.
Reliable AI-supported testing separates candidate generation from evaluation, uses current external evidence where needed and records what result would disconfirm the preferred hypothesis before seeing the outcome.
High-stakes tests still require domain-appropriate methods, authorised data and accountable human interpretation.
14. The Hypothesis Testing Audit
- Alternatives: Which explanations are being separated?
- Prediction: Where do their expectations differ?
- Discriminator: Which observation has the highest information value?
- Reliability: Can the test measure the difference cleanly?
- Scope: Are the test conditions appropriate?
- Ethics: Is the evidence worth obtaining this way?
- Precommitment: How will each outcome change confidence?
- Weight: How strong is the resulting evidence?
- Stopping: When is enough evidence enough for the decision?
- Revision: Does failure reject the hypothesis or merely narrow it?
15. Applying the Idea: Test Explanations Where They Differ
Several hypotheses may plausibly explain the current situation. Hypothesis testing asks which evidence would distinguish their predictions.
A strong test examines conditions under which one candidate explanation predicts a different outcome from another.
A test earns its cost when it helps distinguish the competing explanations.
16. Return to the Decisive Observation
Testing is intelligence deciding which new contact with reality is worth buying.
The strongest test exposes a meaningful disagreement among candidate maps, measures the difference reliably and changes what the system should believe or do next.
More evidence is not always better. Better-discriminating evidence is better.