Choose your reading route
eduKateSG · Why Mathematics? · Brunner–Munzel, Placement Ranks and Probabilistic Superiority
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 5–8
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 9–14
Check assumptions
See where design and calibration matter.Route 4 · Chapters 15–16
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 17–23
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when two groups overlap? Because an average cannot tell us how often a randomly chosen observation from one group exceeds one from another. The Brunner–Munzel test addresses that particular question without demanding that the two populations have identical shapes or variances. Its central object is a probability, not a claim that one group is universally better.
Imagine comparing completion scores from two independent groups. Some pupils in each group do very well, some struggle, and several scores tie. Rather than labelling a whole group stronger, we can ask: if we draw one score from each population, what is the chance the second exceeds the first, counting a tie as half a win? That is a useful example of the benefits of learning mathematics: a sweeping judgement becomes a precise, limited comparison.
This article explains the Brunner–Munzel mechanism, its connection to ranks, a fully reproducible worked example, and what students can learn from it. The data below are illustrative, not an evaluation of a real school or teaching method. The topic is enrichment in statistics and problem-solving, not a claim about a particular Singapore examination syllabus.
Did You Know? Two distributions can differ substantially while neither has a greater than one-half chance of producing the larger observation. Distribution equality and probabilistic superiority are different hypotheses. Understanding that distinction matters more than remembering the test’s name.
Full chapter index
Chapters 1–4 · Understand the question
Chapters 5–8 · Reproduce the example
Chapters 9–14 · Check assumptions
Chapters 15–16 · Explore applications
Chapters 17–23 · Study and verify
CHAPTER 1 OF 23 · Understand the question
1. Start with the probability, not the test
Let X denote an observation from population A and Y one from population B. Define θ=P(X<Y)+½P(X=Y). Throughout this article a larger value is treated as the outcome of interest; for completion time, where lower can be preferable, the practical interpretation must be reversed or the variable redefined transparently.
The null hypothesis is θ=½. It says neither population has a pairwise ordering advantage after ties are split evenly. The two-sided alternative is θ≠½. A directional alternative must be chosen before examining results, based on the research question, not selected because the observed estimate points conveniently one way.
This probability is about two independent draws. It is not the proportion of people who personally improve, a probability that a programme works for an individual, or the fraction of all future situations in which one group wins. Those are different questions requiring different data and assumptions.
Why include half a tie? It makes the comparison symmetric. For any pair, A’s win credit plus B’s win credit equals one. If all values are tied, each side receives one-half. For ordinal ratings this convention is especially useful because ties are common and pretending otherwise would change the measurement.
CHAPTER 2 OF 23 · Understand the question
2. A comparison grid makes the estimand visible
Write A’s observations down the side of a grid and B’s across the top. In each cell enter 1 if Y exceeds X, 0 if it falls below X, and 0.5 if the values tie. The mean of the entire grid estimates θ. With m observations from A and n from B, there are mn comparisons.
For the tiny illustration X=[1,3] and Y=[2,3], the four credits are 1,1,0,0.5. Their sum is 2.5, so the estimate is 2.5/4=0.625. This is a transparent arithmetic description of those observations. It is not yet an adequately calibrated small-sample inference.
The grid also reveals a subtle dependence. Each X value appears in n cells and each Y value in m cells. The mn entries are not mn independent observations. Counting every comparison as an independent trial would exaggerate the available information, sometimes dramatically.
The actual independent sampling units remain the original observations. If scores come from pupils, the units are pupils, not comparison cells. If several scores come from each pupil, even that statement needs revisiting because repeated measurements create another layer of dependence.
| X observation | Y = 2 | Y = 3 |
|---|---|---|
| 1 | 1 | 1 |
| 3 | 0 | 0.5 |
CHAPTER 3 OF 23 · Understand the question
3. Why unequal spread changes the uncertainty
Suppose one group’s values are tightly packed while the other has a broad distribution. An A observation near the middle may beat nearly all B observations on one occasion and few on another. The uncertainty contributed by the two groups can be different even when their sample sizes match.
Brunner–Munzel studentizes the estimated probability using separate estimates of these contributions. In plain language, it divides the departure from one-half by an estimated standard error that respects the two-sample structure. This is related in spirit to allowing unequal variances when comparing means, but the target here is a relative ordering probability.
This does not make the procedure immune to a poor design. Convenience samples, selective exclusions, dependent observations and changing measurement definitions still matter. A method can relax one assumption without removing every assumption.
It is also important not to describe every Mann–Whitney analysis as invalid whenever variances differ. The meaning of its null and calibration depend on the formulation and design. The useful contrast here is narrower: Brunner–Munzel explicitly targets θ=½ while allowing different population distributions under that null.
CHAPTER 4 OF 23 · Understand the question
4. Placement values explain the standard error
For each X observation, calculate its average credit against all Y observations. Call these a₁,…,a_m. For each Y observation, calculate its average credit against all X observations. Call these b₁,…,b_n. Both sets average to the same overall estimate θ̂.
Let s_a² and s_b² be the ordinary sample variances of those two sets of placements, using denominators m−1 and n−1. A transparent expression for the estimated squared standard error is s_a²/m+s_b²/n. Taking the square root produces SE. The studentized statistic is (θ̂−½)/SE, subject to a software sign convention.
Why calculate two variances? Resampling the X population changes the row placements, while resampling Y changes the column placements. The two sources do not have to contribute equally. Keeping them separate is precisely what a naive binomial treatment of the mn cells fails to do.
The approximation uses a degrees-of-freedom calculation analogous to a Welch–Satterthwaite construction. If u=s_a²/m and v=s_b²/n, then ν=(u+v)²/[u²/(m−1)+v²/(n−1)]. A t reference with this ν supplies an approximate p-value. A non-integer ν is entirely legitimate.
CHAPTER 5 OF 23 · Reproduce the example
5. Worked example: 120 comparisons, 22 observations
Use X=[1,2,3,4,5,6,7,8,9,10] and Y=[2,3,4,5,6,7,8,9,10,14,18,22]. There are 10 X observations, 12 Y observations and 120 pairwise comparisons. The groups overlap considerably, while Y has several larger values.
The total Y-favouring credit is 79.5. Therefore θ̂=79.5/120=0.6625. In this sample, a randomly selected cross-group comparison gives Y an estimated 66.25% win-or-half-tie credit. That is a statement about pair ordering, not an estimated 66.25% chance that every Y participant benefits from something.
The row placements are approximately [1,0.95833,0.875,0.79167,0.70833,0.625,0.54167,0.45833,0.375,0.29167]. The column placements are [0.15,0.25,0.35,0.45,0.55,0.65,0.75,0.85,0.95,1,1,1]. Both averages equal 0.6625; that is an excellent bookkeeping check.
Using the placement variances gives SE≈0.11846113 and ν≈19.96811. The statistic is (0.6625−0.5)/0.11846113≈1.37175801. A two-sided t-reference calculation gives p≈0.18535619. An independent calculation with SciPy 1.17.0 reproduced these values using its default t reference.
At a predeclared 5% threshold we do not reject θ=½. The estimate still points toward Y, but the uncertainty is substantial. “The samples prove the groups equal” is wrong; “Y is definitely better because 66.25 exceeds 50” is also wrong. Estimate, uncertainty and decision need separate sentences.
The comparison grid is the clearest teaching representation, but a large grid is expensive to store. Pooled midranks and within-sample ranks can recover the same placement information without materialising every cell. This is an example of mathematics in computing: algebra reorganises a calculation while preserving its target.
A midrank assigns tied observations the average of their occupied rank positions. If three tied values would occupy ranks 4,5,6, each receives rank 5. That is not arbitrary smoothing; it preserves the sum of ranks across the tied block.
Pooled ranks record how an observation sits among both groups. Within-group ranks record how it sits among its own group. Their difference isolates the contribution from the other group, with the appropriate tie convention. SciPy’s implementation uses this structure to obtain the group-specific variability terms.
Students should first verify the shortcut against a small comparison grid. Agreement on one clean example is not sufficient: include ties, unequal sample sizes and an example where the ordering advantage reverses. A shortcut becomes trustworthy when its meaning survives deliberately awkward cases.
CHAPTER 7 OF 23 · Reproduce the example
7. Equal medians do not answer this question
Medians describe a central quantile. The probability θ uses all cross-group comparisons. Depending on distributional shape, populations can share a median yet have a nontrivial ordering pattern, or differ in features that leave θ at one-half. There is no universal conversion between a median difference and probabilistic superiority.
Consider X fixed at zero while Y is equally likely to be −10 or +10. Each population has a central balance around zero, and P(X<Y)=½. The distributions are obviously not identical: one has no spread and the other is broad. This simple population example demonstrates why θ=½ does not imply equality of distributions.
The fixed-X example is useful conceptually but can produce degenerate estimated variance terms in real test calculations. A teaching counterexample about the estimand is not automatically a valid dataset for every approximate test. Keep those two roles separate.
When someone asks “which group is better?”, first decide whether the question means higher mean, higher median, more consistent performance, a larger ordering probability, or more people above a meaningful threshold. The test should follow that choice rather than quietly defining it afterwards.
CHAPTER 8 OF 23 · Reproduce the example
8. Ties are information, not a nuisance to erase
Many educational outcomes are discrete: rubric bands, short quiz marks and five-point ratings. Ties may reflect the actual measurement scale rather than a defect in the data. The half-credit convention allows the relative effect to remain interpretable without inventing tiny differences.
Adding random jitter to break ties changes comparisons that were originally equal. Repeating this with different seeds can change the result. Unless a carefully justified measurement model requires such a step, retain the observed ties and use a procedure that handles them.
Rounding can also create ties artificially. If the instrument measured tenths but a spreadsheet stores integers, preserve the original values when authorised and available. Conversely, adding decimal places to integer observations does not restore missing precision.
An instructive classroom activity is to calculate θ̂ before and after rounding. Ask which pairs changed from wins to ties, and why. The lesson is not that one representation is automatically wrong; it is that a measurement decision changes the mathematical question the data can answer.
Two different spreadsheets do not guarantee two independent samples. The same pupil may appear in both, siblings may share relevant conditions, or several observations may come from one classroom. Independence concerns the sampling or assignment process, not file organisation.
For paired before-and-after measurements, one naturally asks about within-person change. Applying an independent-samples test discards the matching and may misrepresent uncertainty. A paired analysis or a model accounting for that dependence is more appropriate.
For clustered data, such as pupils within classes, the effective independent units may be classes or a hierarchical sampling structure. Comparing hundreds of pupil scores while ignoring a small number of shared classroom environments can create a misleading impression of precision.
Write the sampling unit explicitly in a project: “one independently sampled device”, “one participant”, or “one independently selected batch”. Then draw arrows showing repeated or shared influences. This small design diagram often catches a more consequential error than any final decimal-place check.
SciPy’s current documentation offers a t or normal reference, with t as the default. It also points readers toward a studentized permutation version for very small samples. Those are calibration choices, not three different definitions of the relative effect.
A raw label permutation is exact under identical-distribution exchangeability, which is stronger than θ=½. When distributions differ under the weaker null, one cannot simply assume every relabelling is equally justified. The studentized permutation literature addresses this nonparametric Behrens–Fisher problem more carefully.
This distinction matters because “permutation” is sometimes treated as a universal guarantee. A resampling algorithm needs a null invariance argument, a clearly defined statistic and a treatment of degenerate resamples. Software that shuffles labels can still be testing the wrong null.
For a school enrichment project, it is perfectly acceptable to stop at the comparison probability and a transparent simulation rather than claim a specialist exact test. Intellectual honesty includes recognising when the inferential machinery requires more background than the available study supports.
CHAPTER 11 OF 23 · Check assumptions
11. Practical investigation: a fair comparison of scores
Create two synthetic groups of twelve quiz scores, using the same possible score range. First make their distributions identical. Repeatedly sample new groups and record the estimates. They will not all equal one-half; sampling variability is the expected behaviour, not a failure.
Next give one population a small location shift while retaining substantial overlap. Plot the estimate distribution again. Some samples will still favour the other group. This is a useful antidote to the misconception that a real population difference must be obvious in every small dataset.
Finally change only the spread. Use symmetric populations centred at the same value. Their distributions differ, but the relative effect can remain one-half. Compare a test of distribution equality with a test of θ=½ and explain why different answers can both be reasonable.
Keep the outcomes synthetic and avoid ranking real pupils publicly. The educational objective is to understand evidence, not to attach labels to children. If a real school dataset is used, permissions, privacy and appropriate supervision belong in the project plan.
CHAPTER 12 OF 23 · Check assumptions
12. Practical investigation: runtime benchmarks
Two software versions produce distributions of completion times. Lower time is preferable, so define the target as the probability the new version is faster, with ties split. Do not reuse a higher-score interpretation without changing direction.
Run independent trials under a documented environment, or explicitly model pairing if each version is tested on the same workload. Warm-up, caching, background tasks and hardware changes can introduce systematic variation that an ordering statistic cannot explain away.
Report the pairwise effect alongside latency quantiles and actual time differences. An ordering advantage might be operationally trivial if it amounts to microseconds, or insufficient if rare slow responses are the real concern. A single probability does not summarise every aspect of performance.
This example connects mathematics in careers to concrete responsibilities in computing and engineering: define the desired property, preserve the design, quantify uncertainty and communicate limits. It does not mean learning one test grants access to a career or replaces domain knowledge.
A hypothesis test assesses compatibility with one reference value. An interval attempts to describe plausible values of the effect under a stated method. Reporting θ̂ alone can conceal whether the data distinguish a modest advantage from a substantial one.
A simple normal interval θ̂±1.96SE may be a rough teaching approximation, but it can extend outside [0,1] and perform poorly in small samples or near boundaries. Do not present it as a universally reliable confidence procedure for this problem.
A defensible analysis should choose an interval method appropriate to the estimand, sample size, ties and heterogeneity. The current research literature includes range-preserving and other alternatives. That is a reason to consult a specialist reference, not to quietly clip an invalid interval to zero and one.
For introductory work, label the estimate and standard error separately and state that interval construction is a further methodological choice. An honest partial summary is more useful than a polished but unjustified interval.
An informative result paragraph begins with the target and direction: “We estimated the probability that an independent B observation exceeded an independent A observation, counting ties as half.” It then gives the estimate, sample sizes, calibration and limitations.
For our example: “The estimate was 0.6625 from samples of 10 and 12 observations. The Brunner–Munzel t approximation gave W≈1.372, ν≈19.97 and two-sided p≈0.185. The data did not reject a one-half relative effect at the predeclared 5% level.”
Add the design boundary: these are illustrative independent samples, not evidence that a real intervention caused an improvement. A causal claim needs randomisation or another justified causal identification strategy. Neither a small p-value nor a sophisticated name supplies that strategy.
Avoid phrases such as “18.5% probability the null is true” or “81.5% confidence B is better”. A p-value is not a posterior probability of the hypothesis, and subtracting it from one does not produce the probability the research claim is correct.
CHAPTER 15 OF 23 · Explore applications
15. A student’s route from fractions to inference
Begin with fractions and comparisons. Count wins, ties and losses for a tiny grid. Verify that the total credit for the two directions equals the number of pairs. That identity links probability to arithmetic students can inspect directly.
Then learn averages and sample variance. Calculate the placement mean and variance by hand for a short example. Explain why averaging the row placements and averaging the column placements must give the same overall effect.
Next connect standard error to repeated sampling. A simulation can show that estimates vary even when the population is unchanged. Compare the observed simulation spread with the estimated standard errors, while acknowledging that simulation checks only the chosen scenarios.
Only then introduce t references, degrees of freedom and p-values. Their role is calibration, not a replacement for understanding the comparison. Students who can explain the grid but not yet derive the approximation have already learned something substantial about numeracy and statistical reasoning.
Ask the learner what is being compared before asking whether the result is significant. “A randomly drawn pair” and “the same child before and after” sound similar in everyday language but imply different mathematical designs. Being able to explain the difference is genuine progress.
Use neutral synthetic contexts so a statistics lesson does not become a judgement about a child’s ability. The strongest teaching question is often “what would this method miss?” A learner who identifies rare extreme outcomes, dependence or practical magnitude has understood the method more deeply than someone who only runs a command.
Build on secure foundations rather than accelerating into terminology. Fractions, ordering, algebra, functions and careful graph reading all support advanced statistics. Enrichment can develop curiosity and disciplined explanation without promising intelligence, results, university entry or employment.
If a student makes a sign error, return to one obvious pair: X=1 and Y=10. Which direction should receive the win? A concrete check can repair the reasoning faster than memorising a software alternative label. Confidence grows when errors become diagnosable rather than mysterious.
Question 1: X=[1,2] and Y=[2,3]. What is θ̂? The credits are 1,1,0.5,1, giving 3.5/4=0.875. The tie belongs to X=2,Y=2. Do not count it as a full win or silently discard it.
Question 2: Swap the groups. What happens? The effect becomes 1−0.875=0.125. This complement check should hold whenever the same half-tie convention is used. A two-sided inference should not change merely because the group labels were swapped.
Question 3: Add 100 to every observation in both groups. The pairwise comparisons remain unchanged, so θ̂ is unchanged. Apply a strictly increasing transformation and the same ordering property holds. A non-strict transformation such as rounding can introduce new ties and need not preserve the value.
Question 4: Are four comparison cells four independent observations? No. There are two X and two Y observations. Reusing each one in multiple cells creates dependence. Explain this before attempting any variance calculation.
Question 5: If p=0.20, is the relative effect equal to one-half? Not established. The test has not rejected that value under its calibration and chosen threshold. The estimate, uncertainty and sampling limitations still need reporting.
Perfect separation can make the approximation degenerate
Suppose every Y observation exceeds every X observation. The estimate is one, and all row and column placements are constant. Their sample variances are zero, so the usual studentized calculation divides by a zero standard error. A software warning here is not evidence that the ordering was unobserved; it signals that this particular approximate calibration has reached a boundary.
Report the observed separation and sample sizes, then use an inferential treatment justified for that setting. Do not replace the undefined output with p=0 or claim absolute certainty about future populations. A finite sample can be perfectly separated even when population overlap exists. Boundary estimates require particular care precisely because a simple variance approximation can become overconfident.
One extreme value changes a different feature
Replace Y’s largest value, 22, with 2200 in our worked example. It still exceeds every X value, so all of its comparison credits remain one. The estimated ordering probability and Brunner–Munzel calculation remain unchanged. The sample mean, however, changes greatly.
This is neither an automatic strength nor a defect. If the scientific question concerns a usual pairwise ordering, the invariance is appropriate. If an extreme cost of 2200 matters to a budget, the ordering probability misses information the decision needs. A robust ordering summary should not be advertised as a complete risk assessment. Always retain a complementary description of magnitude and tails.
Unequal sample sizes do not create more independent pairs
Hold ten X observations fixed and collect another hundred independent Y observations. The estimate of Y’s distribution improves, but information about X is still limited by ten observations. The two placement-variance terms show why increasing only one group cannot necessarily remove most uncertainty.
This is useful when planning a study with unequal collection costs. More data from the cheap group may help less than a modest increase in the expensive, poorly estimated group. A realistic power or precision study should vary both sample sizes and the expected distributional shapes. It should not infer design adequacy from the product mn, which counts comparisons rather than independent sampling units.
A mixture can hide opposing subgroups
Suppose a synthetic population combines two settings in which the relative ordering points in opposite directions. The aggregate θ may be near one-half even though each setting has a meaningful contrast. That does not refute the aggregate calculation; it shows that the population being averaged over matters.
Record relevant composition and consider a preplanned stratified or adjusted analysis when the question requires it. Do not search dozens of subgroups after a non-significant result and report only a favourable one. Distinguishing a meaningful prior subgroup from a discovered pattern is part of the inferential task, and all results should remain connected to the sampling design.
Build a small comparison calculator with three outputs: the grid of win-or-half-tie credits, its overall average, and the row and column placement summaries. Include a clear sentence defining which direction receives the win. A transparent spreadsheet is sufficient; a polished interface is unnecessary.
Test five manufactured inputs: identical samples, swapped group labels, all values tied, perfect separation and a dataset with one extreme observation. Predict each effect estimate before running the calculator. Mark which cases make the approximate standard error degenerate rather than silently forcing a numerical result.
Then compare a grid implementation with a rank-based implementation on a set of nondegenerate examples. Differences should be explained through tie handling, sample order or a coding error. Agreement is an implementation check, not proof that either method answers a real-world causal question.
End the project with a short decision note. Name a situation where pairwise ordering is useful and another where it is insufficient. For example, routine performance comparison may benefit from an ordering probability while safety decisions also need worst-case or tail information. This final contrast demonstrates transfer more convincingly than a list of impressive statistical terms.
- Define X, Y and the direction of a favourable outcome before calculating.
- State the independent sampling unit and identify any pairing or clustering.
- Report θ̂ with sample sizes and the treatment of ties.
- Name the calibration, software version and missing-data policy.
- Retain raw-unit summaries that reveal magnitude, spread and important tails.
- Explain whether the study supports association, description or a causal interpretation.
- Describe boundary warnings and any sensitivity analysis rather than hiding them.
- Separate exploratory comparisons from prespecified hypotheses.
The checklist is deliberately about meaning as well as computation. A technically correct statistic can still be an unhelpful answer if the direction was reversed, the units were misdescribed or the practical target was a different property. That is why statistical reasoning belongs alongside calculation in mathematics education.
Does Brunner–Munzel compare means?
No. Its target is a probabilistic ordering effect with half credit for ties. Means may be scientifically important, but they require an analysis targeting means. Choosing a rank-based procedure changes the question; it is not just an automatic repair for an inconvenient normality test.
Can the two groups have different variances?
Yes, variance equality is not the requirement that defines this test. The procedure estimates the two placement-variability contributions separately. Independence and reliable calibration still matter, and extreme small-sample situations can remain difficult.
Does θ=½ mean the distributions are identical?
No. Equal distributions imply balanced pairwise ordering, but the reverse need not hold. Differences in spread or shape can cancel in this particular probability. Use a distribution-equality procedure when that broader null is the scientific question.
Are ties allowed?
Yes. The relative effect includes half the tie probability, and midrank-based calculations preserve this convention. Do not remove ties by arbitrary jitter or claim that ordinal ratings contain more precision than they actually have.
Is it appropriate for the same people measured twice?
Not as an independent two-sample analysis. Matching creates dependence and usually changes the question to within-person differences. A paired or repeated-measures method should preserve that design.
Which software setting should I record?
Record the implementation and version, group order, alternative, t or normal reference, missing-data policy and any resampling procedure. A reproducible result needs more than a test name and a p-value.
What is the main educational benefit?
The method teaches students to distinguish an estimand from a statistic, information from repeated bookkeeping, and uncertainty from indecision. Those distinctions transfer to reading research, evaluating technology and making careful everyday comparisons.
CHAPTER 22 OF 23 · Study and verify
22. One last distinction: probability is not a personal label
A relative effect describes a population comparison under a defined sampling process. It does not divide people into winners and losers or identify which individual should choose an option. Two populations can overlap widely even when the ordering probability differs from one-half.
For a personal decision, combine relevant population evidence with the individual’s circumstances, preferences and constraints. In education, a group statistic cannot diagnose one learner’s missing prerequisite or predict a guaranteed result. A student’s actual work, explanations and learning history provide different information.
This is a constructive boundary, not a rejection of quantitative evidence. Mathematics is most helpful when it clarifies one part of a decision while leaving room for other justified evidence. Knowing what a probability refers to is the first step toward using it humanely and accurately.
The foundational reference is Brunner and Munzel’s 2000 paper, The Nonparametric Behrens–Fisher Problem. The operational interface and calibration cautions are described in SciPy’s Brunner–Munzel documentation. The worked numbers here were independently computed rather than copied from its example.
For a different target, read Ansari–Bradley scale comparisons: it asks about dispersion under its own model, not this ordering probability. For the design contrast, McNemar’s matched binary comparison keeps paired observations together.
Continue through the eduKateSG Mathematics hub to connect statistics with algebra, probability and mathematical modelling. The lasting lesson is simple: mathematics makes a comparison useful by saying exactly what it means, how uncertain it is, and what it cannot establish.