Chapter map

1. 1. Question
2. 2. Ordering
3. 3. Labels
4. 4. Sequence
5. 5. Run boundary
6. 6. Observed runs
7. 7. Label totals
8. 8. Expected runs
9. 9. Variance
10. 10. Standardization
11. 11. Exact enumeration
12. 12. Lower tail
13. 13. Upper tail
14. 14. Threshold sensitivity
15. 15. Missingness
16. 16. Attribution

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Question

This is where modelling choices begin to affect evidence. Decide whether testing two-sample mixing or sequence randomness.

A concrete situation

A binary time sequence is examined.

What the mathematics does not license

The interpretations differ. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Name the design. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. Ordering

The central mathematical idea is concrete. Fix chronological or sorted order as the design requires.

A concrete situation

Eight labels are time ordered.

What the mathematics does not license

Changing order changes runs. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Freeze it. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Labels

Here the geometry becomes operational. Define A and B without outcome tuning.

A concrete situation

A and B each occur four times.

What the mathematics does not license

A median split can create ties. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Declare threshold. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Sequence

This step deserves a careful pause. Write every label in order.

A concrete situation

A,A,B,B,A,B,B,A.

What the mathematics does not license

Transcription errors alter boundaries. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Audit it. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Run boundary

The useful question is not only how to calculate, but what the calculation assumes. Mark each label change.

A concrete situation

AA|BB|A|BB|A.

What the mathematics does not license

A run is not one observation. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Use maximal blocks. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Observed runs

A small example makes the mechanism visible. Count the blocks.

A concrete situation

R=5.

What the mathematics does not license

The count alone has no scale. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Compute its null. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Label totals

This is where modelling choices begin to affect evidence. Count n1 and n2.

A concrete situation

n1=n2=4.

What the mathematics does not license

Highly imbalanced totals limit possible runs. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Report both. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Expected runs

The central mathematical idea is concrete. Calculate 1+2n1n2/n.

A concrete situation

Expected R=5.

What the mathematics does not license

This is conditional on totals. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

State conditioning. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Variance

Here the geometry becomes operational. Use the finite-sample formula.

A concrete situation

Variance=768/448≈1.714.

What the mathematics does not license

Approximations are weak at n=8. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Prefer exact inference. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Standardization

This step deserves a careful pause. Compare observed with mean using the standard deviation.

A concrete situation

Observed minus expected is zero.

What the mathematics does not license

Continuity corrections vary. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Declare convention. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Exact enumeration

The useful question is not only how to calculate, but what the calculation assumes. Enumerate all arrangements with fixed totals.

A concrete situation

There are choose(8,4)=70 arrangements.

What the mathematics does not license

Two-sided definitions can differ. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Archive tail counts. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Lower tail

A small example makes the mechanism visible. Interpret unusually few runs as clustering.

A concrete situation

Long A and B blocks reduce R.

What the mathematics does not license

A change point can create clustering. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Compare break models. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Upper tail

This is where modelling choices begin to affect evidence. Interpret unusually many runs as alternation.

A concrete situation

ABABABAB has eight runs.

What the mathematics does not license

Periodic mechanisms may explain it. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Inspect timing. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Threshold sensitivity

The central mathematical idea is concrete. Vary a justified label threshold.

A concrete situation

Near-threshold observations switch labels.

What the mathematics does not license

Outcome-selected thresholds bias results. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Publish sensitivity. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. Missingness

Here the geometry becomes operational. Preserve adjacency honestly.

A concrete situation

A missing time lies between labels.

What the mathematics does not license

Deleting it joins observations that were not adjacent. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Declare gaps. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Attribution

This step deserves a careful pause. Use domain evidence to explain patterns.

A concrete situation

A process change matches a cluster.

What the mathematics does not license

Runs do not prove cause. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Validate independently. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

The Wald–Wolfowitz runs test studies how two labels are arranged after observations are ordered or how a binary sequence alternates. A run is a maximal uninterrupted block of one label. Too few runs indicate clustering; too many indicate alternation. Conditioning on the number of each label gives an exact combinatorial null.

If there are n1 labels of type A and n2 of type B, the expected run count is 1+2n1n2/(n1+n2). The usual variance is 2n1n2(2n1n2-n1-n2)/[(n1+n2)^2(n1+n2-1)]. Exact enumeration is preferred when counts are small.

For A,A,B,B,A,B,B,A there are five runs: AA | BB | A | BB | A. Here n1=n2=4, so the expected number is 1+2(4)(4)/8=5. The variance is 2(4)(4)(32-8)/(64×7)=768/448, about 1.714. The observed sequence sits exactly at the null mean.

A runs test discards distances and magnitudes after labels are formed. A median split can create ties and a data-dependent threshold; the two-sample version answers a distributional ordering question rather than a time-trend question. Responsible analysis states how labels and order were chosen before interpreting the run count.

---

Frequently asked questions

What is a run?

It is a maximal consecutive block containing only one of the two labels.

What do too few runs suggest?

They suggest clustering or positive persistence relative to the conditional random-order null.

What do too many runs suggest?

They suggest alternation or negative persistence.

Why condition on label totals?

The exact null randomizes order while keeping the observed counts of A and B fixed.

How are ties at a median split handled?

The analyst must declare exclusion, deterministic assignment or another rule before testing.

Does the runs test use magnitudes?

No. Once labels are formed, magnitudes and distances from the threshold are discarded.

---

Primary source and fact-check note

Abraham Wald and Jacob Wolfowitz, “On a Test Whether Two Samples are from the Same Population,” The Annals of Mathematical Statistics 11(2), 147–162 (1940), DOI 10.1214/aoms/1177731909. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• Bartels rank test
• von Neumann ratio test
• How Mathematics Works

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: topic-specific validation and responsible use

Design audit

Separate two-sample and sequence questions.

Label audit

Publish thresholds and tie rules.

Order audit

Freeze the sequence.

Run audit

Mark every boundary visibly.

Null audit

Condition on label totals.

Exact audit

Enumerate small samples.

Sensitivity audit

Vary justified thresholds.

Final claim

Describe clustering or alternation only.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Question

Begin with this mechanism: Decide whether testing two-sample mixing or sequence randomness. Recreate the concrete case: A binary time sequence is examined. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: The interpretations differ. Complete the transfer step: Name the design. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 2: Ordering

Begin with this mechanism: Fix chronological or sorted order as the design requires. Recreate the concrete case: Eight labels are time ordered. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Changing order changes runs. Complete the transfer step: Freeze it. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 3: Labels

Begin with this mechanism: Define A and B without outcome tuning. Recreate the concrete case: A and B each occur four times. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: A median split can create ties. Complete the transfer step: Declare threshold. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 4: Sequence

Begin with this mechanism: Write every label in order. Recreate the concrete case: A,A,B,B,A,B,B,A. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Transcription errors alter boundaries. Complete the transfer step: Audit it. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 5: Run boundary

Begin with this mechanism: Mark each label change. Recreate the concrete case: AA|BB|A|BB|A. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: A run is not one observation. Complete the transfer step: Use maximal blocks. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 6: Observed runs

Begin with this mechanism: Count the blocks. Recreate the concrete case: R=5. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: The count alone has no scale. Complete the transfer step: Compute its null. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 7: Label totals

Begin with this mechanism: Count n1 and n2. Recreate the concrete case: n1=n2=4. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Highly imbalanced totals limit possible runs. Complete the transfer step: Report both. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 8: Expected runs

Begin with this mechanism: Calculate 1+2n1n2/n. Recreate the concrete case: Expected R=5. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: This is conditional on totals. Complete the transfer step: State conditioning. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 9: Variance

Begin with this mechanism: Use the finite-sample formula. Recreate the concrete case: Variance=768/448≈1.714. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Approximations are weak at n=8. Complete the transfer step: Prefer exact inference. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 10: Standardization

Begin with this mechanism: Compare observed with mean using the standard deviation. Recreate the concrete case: Observed minus expected is zero. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Continuity corrections vary. Complete the transfer step: Declare convention. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 11: Exact enumeration

Begin with this mechanism: Enumerate all arrangements with fixed totals. Recreate the concrete case: There are choose(8,4)=70 arrangements. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Two-sided definitions can differ. Complete the transfer step: Archive tail counts. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 12: Lower tail

Begin with this mechanism: Interpret unusually few runs as clustering. Recreate the concrete case: Long A and B blocks reduce R. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: A change point can create clustering. Complete the transfer step: Compare break models. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 13: Upper tail

Begin with this mechanism: Interpret unusually many runs as alternation. Recreate the concrete case: ABABABAB has eight runs. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Periodic mechanisms may explain it. Complete the transfer step: Inspect timing. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 14: Threshold sensitivity

Begin with this mechanism: Vary a justified label threshold. Recreate the concrete case: Near-threshold observations switch labels. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Outcome-selected thresholds bias results. Complete the transfer step: Publish sensitivity. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 15: Missingness

Begin with this mechanism: Preserve adjacency honestly. Recreate the concrete case: A missing time lies between labels. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Deleting it joins observations that were not adjacent. Complete the transfer step: Declare gaps. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.

Workshop 16: Attribution

Begin with this mechanism: Use domain evidence to explain patterns. Recreate the concrete case: A process change matches a cluster. Write ordered labels, run boundaries, run counts, label totals, possible arrangements, expected runs, variances, standardized values, exact tail counts, thresholds and missing positions with units, denominators and assumptions. Calculate the smallest non-trivial example by hand and reproduce it in a spreadsheet or short program. Challenge this limitation: Runs do not prove cause. Complete the transfer step: Validate independently. Change exactly one observation, timestamp, rank, tie rule, half-pairing rule, label threshold, run boundary, lag, null distribution, tail convention, simulation seed, reference period or validation window. Explain which mathematical object changes, which conclusion survives and which claim must weaken. Keep observation, preprocessing, statistic, calibration, uncertainty, interpretation and independent validation on separate notebook lines. Reserve complete periods outside tuning when testing transfer.