The Gold Standard of Universities | When Cohort Mix Reverses an Outcome Comparison

A headline rate can answer how many people in a defined observed group met a stated criterion. It cannot automatically answer whether one university caused better outcomes, whether one programme suits a particular student or what would happen if the institutions enrolled identical populations.

Start with the intended question. Are you describing the actual graduating group? Comparing like groups? Investigating a change across years? Choosing a programme whose daily learning matches your interests? These questions can require different evidence even when they use some of the same tables.

In this guide, an outcome is simply a binary fictional criterion: a person either meets it or does not meet it at the stated observation point. We deliberately avoid attaching an employment definition to the invented values. Real surveys define employment, further study, response populations and timing in specific ways.

That separation makes the mathematical question easier to inspect. First understand how a percentage is formed. Then ask whether its real-world definition fits your decision. A sophisticated adjustment cannot repair a mismatch between the outcome being measured and the educational quality you hoped to understand.

Write a plain sentence before a formula: I want to compare this outcome among these people, at this point, under these definitions. If the sentence remains vague, collecting more decimal places will not fix it. The university root supplies the wider questions about curriculum, assessment, support and institutional responsibility that should accompany any numerical comparison.

Contents · Next chapter

A rate such as 80% needs a numerator and denominator. In a simple binary exercise, the numerator counts people meeting the criterion and the denominator counts the eligible people whose outcomes are included. Keep those definitions attached to both counts.

Suppose a fictional group has four successes among five people. Its rate is four divided by five, or 80%. Another has sixty among 100, or 60%. Combining the groups gives sixty-four successes among 105 people, approximately 61.0%. Averaging 80% and 60% gives 70%, which treats the groups as equally large.

Neither arithmetic operation is intrinsically meaningless. They answer different questions. The pooled rate describes the combined people. The equal-group mean describes a hypothetical weighting that gives each group half the influence. It is misleading to label the latter as the actual combined proportion.

Require counts before combining percentages. If a table supplies only rounded rates, exact pooled reconstruction may be impossible. You may need the underlying counts or a published combined measure. Do not invent a denominator because a familiar-looking percentage encourages a quick calculation.

This is the same structural idea as credit weighting in the Universities root, but the objects differ. Module credits weight grade contributions; group sizes weight binary person-level outcomes. Learning transfers when the student can identify the relevant weight in the new problem, rather than automatically averaging every pair of percentages that appears on a page.

Contents · Previous chapter · Next chapter

Call the fictional institutions Harbour University and Ridge University. Each has 500 observed people in the exercise. We divide them into two consistently defined groups, X and Y. These neutral labels are deliberate: neither group represents a real subject, social group or level of ability.

At Harbour, ninety of 100 people in X meet the outcome criterion, a rate of 90%. In Y, 240 of 400 meet it, a rate of 60%. At Ridge, 320 of 400 people in X meet the criterion, a rate of 80%. In Y, fifty of 100 meet it, a rate of 50%.

Within X, Harbour is ten percentage points higher. Within Y, Harbour is also ten percentage points higher. Those are direct descriptions of the fictional table, under its matching definitions. They do not establish why the differences exist.

Before reading on, predict the overall comparison and explain your reasoning. If you say Harbour must have the higher pooled rate because it leads in both groups, the next chapter identifies the missing step. The table is designed to expose a common assumption about aggregation, rather than test whether you can spot an arithmetic trick quickly.

Contents · Previous chapter · Next chapter

Harbour has ninety plus 240, or 330 people meeting the criterion among 500. Its pooled rate is 66%. Ridge has 320 plus fifty, or 370 among 500, giving 74%. Ridge’s pooled rate is eight percentage points higher, even though Harbour’s rate is higher within each group.

The explanation lies in the composition. Only 20% of Harbour’s observed people belong to X, the group with the higher outcome rate in both institutions. At Ridge, 80% belong to X. Ridge’s total therefore places much more weight on the higher-rate group.

Write the calculation as a weighted sum. Harbour’s rate is 0.2 times 90% plus 0.8 times 60%, or 18% plus 48%, giving 66%. Ridge’s rate is 0.8 times 80% plus 0.2 times 50%, or 64% plus 10%, giving 74%.

No person’s outcome changed during the calculation. We changed only the level at which the records were summarised. This kind of reversal is associated with Simpson’s paradox. The Office for National Statistics explanation illustrates why population composition matters in interpreting apparently conflicting summaries. Our university table is an original teaching construction, separate from the ONS example.

Contents · Previous chapter · Next chapter

Both pooled rates accurately describe their respective observed groups. It would be wrong to call Ridge’s 74% fabricated merely because the subgroup comparison points the other way. The problem begins when someone uses that descriptive total to answer a different question without acknowledging the group mix.

Similarly, Harbour’s subgroup advantage does not make its actual pooled result 74% or 75%. Its observed combined rate remains 66%. Another measure can help with another comparison, but it should not overwrite the original account.

Imagine a reader asks how many people met the criterion across the actual included population. Use the pooled counts. If the reader instead asks what the comparison looks like at an identical X–Y mix, construct and label a common-weight comparison. If they ask which university caused better outcomes, neither calculation alone settles causation.

This distinction makes discussion calmer and more productive. A disagreement about which summary to use may reveal different questions rather than a mathematical error. Name the question, retain both valid summaries and explain what each contributes.

For a learner check, ask which statement is unsupported: Ridge has the higher pooled rate in this table; Harbour has higher rates within both groups; Ridge must provide better teaching. The first two follow from the records. The third requires evidence about teaching and other influences that the table does not isolate. Identifying that inferential jump is more useful than memorising the name of a paradox.

Contents · Previous chapter · Next chapter

Choose a hypothetical reference population with half its people in X and half in Y. Apply those same weights to both universities. Harbour’s common-weight rate is half of 90% plus half of 60%, or 75%. Ridge’s is half of 80% plus half of 50%, or 65%.

These are constructed comparison measures. They describe what the weighted combination of the observed subgroup rates gives at the chosen mix. They are not new observed outcomes and do not claim that changing enrolment would leave all other conditions unchanged.

The calculation answers a narrower question: how do the reported subgroup rates compare when composition is held to this reference? Harbour leads by ten percentage points. That is consistent with its ten-point advantage in each group.

Why use half and half? In this fictional combined dataset, X totals 500 people and Y totals 500, so the combined population happens to provide equal weights. Another task could choose a different legitimate reference. State the choice and its reason instead of presenting equal weighting as automatically fair for every comparison.

A useful report shows the actual pooled rate, the subgroup table and the reference-weight result together. The reader can then see both the real observed composition and the standardised comparison. Hiding the weights would merely replace one opaque headline with another. The gold-standard habit is to expose the construction clearly enough that someone else can reproduce it and decide whether it suits the question.

Contents · Previous chapter · Next chapter

Apply Harbour’s actual mix, 20% X and 80% Y, to both institutions. Harbour remains at 66%. Ridge’s common-weight result becomes 0.2 times 80% plus 0.8 times 50%, or 56%. The difference remains ten percentage points.

Now use Ridge’s mix, 80% X and 20% Y, for both. Harbour’s result becomes 84%, while Ridge remains at 74%. Again Harbour leads by ten points. The numerical levels depend on the reference mix, while the difference stays constant in this specially constructed example.

That stability happens because Harbour’s advantage is exactly ten points in both groups. For any shared weights adding to one, the weighted difference is ten times the first weight plus ten times the second, still ten. This is a property of the example, not a promise about all datasets.

Change the subgroup differences and the conclusion may depend more strongly on the chosen reference. If one institution leads in X and the other in Y, there may be a mix at which their weighted results are equal. Different plausible reference populations can then yield different rankings.

Use that possibility constructively. It tells a prospective student to examine the relevant programme or subgroup instead of insisting on one universal institutional order. It also tells an analyst to explain sensitivity to reference weights. A result that changes under reasonable choices is still informative; its dependence belongs in the account rather than being hidden behind an authoritative-sounding adjusted score.

Try a separate hypothetical case in which institution A has rates of 90% in X and 40% in Y, while B has 80% in X and 60% in Y. A leads in X but trails in Y. Let the common reference share in X be w, with the remaining share one minus w in Y.

A’s weighted rate is 40 plus fifty times w, expressed as a percentage. B’s is sixty plus twenty times w. Their difference is thirty times w minus twenty percentage points. They tie when w is two thirds. At an equal mix, A gives 65% and B 70%; at 80% X, A gives 80% and B 76%.

Unlike Harbour’s consistent subgroup advantage, this case has no reference-independent ordering across all possible shared mixtures. The numerical answer changes because the reference population gives different influence to the subgroup in which each institution leads. Choosing weights after seeing which ones favour an institution would therefore conceal an important decision.

For an independent explanation, identify which subgroup a student’s question concerns, whether an overall common-mix comparison is useful at all, and which reference population can be defended. The exercise does not ask you to find a weight that makes your preferred institution win. It asks you to connect a comparison method with a clear question and state the method’s dependence on that choice.

Contents · Previous chapter · Next chapter

Composition can also reverse a time comparison within one institution. In fictional Year One, 400 people are in X and 100 in Y. X has an 80% rate, giving 320 outcomes; Y has a 40% rate, giving forty. The pooled rate is 360 among 500, or 72%.

In Year Two, the institution has 100 people in X and 400 in Y. X improves to 85%, giving eighty-five outcomes. Y improves to 50%, giving 200. Yet the pooled rate becomes 285 among 500, or 57%.

The overall rate fell fifteen percentage points while both subgroup rates improved. The mix shifted strongly towards Y, whose rate remains lower. A headline claiming that every part of the institution deteriorated would therefore contradict the subgroup evidence.

At a common half-and-half mix, Year One gives 60% and Year Two 67.5%. That constructed measure isolates the change in these two observed group rates under fixed weights. It does not explain the cause of either improvement. Keep calendar conditions, measurement definitions and the actual composition change available alongside it. The lesson is to investigate the parts before turning one movement in an aggregate into a complete story about institutional progress.

Another check separates two changes instead of compressing them into one headline. Hold Year One’s group rates at 80% and 40%, but apply Year Two’s 20:80 mix. The resulting rate is 48%. Relative to Year One’s actual 72%, this particular ordering assigns a twenty-four-point decrease to the composition change. Then applying Year Two’s rates raises 48% to its actual 57%, a nine-point increase from the rate changes at Year Two’s mix. Together, minus twenty-four plus nine equals the observed minus fifteen points.

The order of this decomposition matters. You could first change the rates at Year One’s mix and then change the mix, producing different component allocations while preserving the same total change. Do not present one ordering as the unique causal explanation. It is an accounting exercise that makes two influences visible under a stated convention.

For a learner, the useful achievement is checking that the pieces reconcile and explaining why the convention must be stated. A transparent decomposition can organise questions for further investigation without pretending to identify everything that made the observed outcomes change.

Contents · Previous chapter · Next chapter

Breaking a dataset into groups does not automatically make a comparison better. The categories should be meaningful for the question, consistently defined and supported by enough information. Selecting a division only because it produces a preferred ranking is poor reasoning.

For a real university question, relevant differences might include programme, qualification level, mode of study or observation timing. Which matters depends on the outcome and purpose. The neutral X and Y groups in the fictional exercise stand in for a clearly defined classification, not a claim that any particular division must always be used.

Avoid treating a broad category as homogeneous. Two programmes grouped under one subject heading may have different structures or professional pathways. A subgroup comparison can still conceal differences inside its own boundary. Deeper analysis needs appropriate data and expertise, rather than endless splitting without a purpose.

There is also a practical limit. Very small cells can create unstable percentages or confidentiality concerns. A public table may suppress them. Respect that suppression instead of trying to recover individual records through subtraction from totals.

A learner can practise by naming a candidate grouping and explaining what it would clarify, what it might conceal and what information is needed. The answer should relate to the original question. Grouping is a modelling choice that deserves a reason; it is not a decorative extra added after the headline calculation has already settled the conclusion.

Contents · Previous chapter · Next chapter

The full eligible cohort and the survey respondents are different populations. An institution may have a particular subject mix among all graduates and another mix among those who answer. A common-weight comparison based on respondents does not automatically repair missing outcomes.

Suppose 200 people are eligible, but only 100 provide usable answers. Eighty respondents meet a fictional criterion. The respondent rate is 80%. If none of the 100 nonrespondents meets it, the full-cohort rate is forty percent. If all do, it is ninety percent. The observed answers alone permit that wide range.

Those are extreme logical bounds under the binary exercise, not estimates that either extreme is likely. They demonstrate why a response rate cannot be ignored when moving from respondents to a whole cohort. Additional information or justified assumptions would be needed to narrow the account.

Now imagine respondents have the same X–Y proportions as the full cohort. That agreement is useful but does not prove representativeness within X or within Y. Nonrespondents could still have different outcomes from respondents inside both groups.

The existing graduate tracer studies guide owns the wider survey and follow-up system. Here, the important distinction is between two weighting problems: differing composition in the population you want to compare, and differing chances that its members appear in the observed evidence. They can interact, but one should not be presented as a complete solution to the other.

Contents · Previous chapter · Next chapter

A comparison can fail before any weighting begins if its outcome definitions differ. Working at the survey date, having secured a position and participating in further study are different conditions. Read the actual report’s notes rather than assume that a shared word such as employment makes every column equivalent.

The MOE Graduate Employment Survey dataset provides an official Singapore route into survey context, calendars, response information and indicator definitions. Use the notes accompanying the relevant release and institution. This article supplies no current salary ranking or substitute survey definition.

Keep three dates separate: the cohort’s completion period, the observation point and the publication date. A table published later may describe an earlier group’s situation. Comparing figures without matching their observation conditions can combine different stages of transition.

Even the same number of months after study can occur under different labour-market conditions. A recession, recruitment cycle or professional training stage may affect an outcome without establishing a change in teaching quality. A common subject mix does not hold those conditions constant.

For a practical check, make a comparison sheet with a definition field and a timing field before a percentage field. If one of the first two is unresolved, mark the numerical comparison as provisional. This keeps the analysis grounded in what the measure actually describes rather than allowing a precise weighted calculation to conceal an uncertain input.

Contents · Previous chapter · Next chapter

The pooled binary rates in this guide can be reconstructed from counts. Medians behave differently. A median describes a position in an ordered set; subgroup medians and subgroup sizes generally do not contain enough information to reconstruct the combined median.

Consider two fictional three-person score sets: one contains 1, 2 and 100; the other contains 3, 4 and 5. Their medians are 2 and 4. Averaging the medians gives 3. But the combined ordered scores are 1, 2, 3, 4, 5 and 100, whose median is the average of the central 3 and 4, or 3.5.

The equal subgroup sizes do not rescue the average-of-medians method. The position of observations around the middle matters. Replacing the first set with another set having the same median can change the pooled ordering while leaving the published subgroup median unchanged.

This matters when a university table places outcome rates beside salary medians. A valid weighted-rate procedure should not be copied across columns merely because both contain numbers. Each statistic has a construction rule.

The ONS article linked earlier explicitly notes the additional complication of medians in its own example. Our small score sets isolate the principle without using actual earnings. For a real pooled median, use the official published result or appropriately accessible underlying data and a suitable method. If the available summary does not identify it, the honest answer is that the combined median cannot be recovered from those summaries alone.

Contents · Previous chapter · Next chapter

Common weighting can hold selected observed categories to the same mix. It does not hold every relevant influence constant. Prior preparation, resources, selection, location, programme differences and other conditions may still differ within the chosen groups.

Thus Harbour’s ten-point common-weight advantage does not establish that attending Harbour would improve a person’s outcome by ten points. That statement asks about what would happen to a person under different institutional experiences, a causal question beyond the descriptive table.

Nor should an adjusted measure be treated as permission to discount the real people represented by the pooled result. The actual cohort’s experience matters. Adjustment is a way of examining a specified comparison, not a declaration that some students count less as people.

An original diagnostic question helps: what changed when we calculated 75% for Harbour? We did not move students, change courses or observe new results. We combined existing group rates using a hypothetical half-and-half reference. The measure’s interpretation should stay within that operation.

For a university choice, use outcome evidence alongside curriculum, assessment, support, access and practical conditions. The course-salary decision guide develops that broader judgement. A carefully adjusted number can improve one part of the discussion while leaving the central question of personal programme fit open for direct investigation.

Contents · Previous chapter · Next chapter

A useful numerical account needs the subgroup counts, outcome counts, definitions, reference weights and calculation. It should identify the observed pooled result and the constructed comparison in different rows or clearly labelled sentences.

For the main exercise, write that Harbour’s actual pooled rate is 66% and Ridge’s is 74%; Harbour has higher observed rates within X and Y; at an equal X–Y reference mix, the weighted rates are 75% and 65%. Then state that these descriptions do not isolate an institutional effect.

Keep rounding until the reporting stage where possible. Multiplying already rounded subgroup rates can introduce discrepancies against exact pooled counts. If source tables supply only rounded figures, describe the reconstruction as approximate and avoid attributing a small difference to an important real-world effect without further support.

Show a sensitivity result when reference weights are consequential. If a reasonable alternative changes the ordering, explain why. If the ordering remains stable across the considered choices, identify which choices were actually tested. Do not claim universal robustness from two convenient examples.

Finally, include the question the comparison was designed to answer. A reader should not have to guess whether the table describes actual graduates, a hypothetical common mix or a prediction. Clear writing is part of statistical quality because it preserves the relationship between the operation performed and the conclusion being offered.

Contents · Previous chapter · Next chapter

Use fictional colleges L and M. L has thirty-six successes among forty people in group P, and forty-eight among eighty in group Q. M has sixty-eight among eighty in P, and twenty among forty in Q. Calculate within-group rates, actual pooled rates and equal-group weighted rates.

L’s rates are 90% and 60%. M’s are 85% and 50%. L leads in both groups. Its pooled rate is eighty-four among 120, or 70%. M’s is eighty-eight among 120, approximately 73.3%. The differing mix reverses the pooled ordering again.

At equal weights, L gives 75% and M 67.5%. The gap is 7.5 percentage points. Unlike the main example, L’s subgroup advantages are five points and ten points, so changing the reference mix changes the size of the weighted gap.

Now use 75% P and 25% Q. L gives 82.5%; M gives 76.25%, a difference of 6.25 points. Explain why it is smaller: the reference places more influence on P, where L’s advantage is only five points.

For the transfer explanation, identify the group proportions, select the weights and name the kind of result. A student who can reproduce a percentage but calls the common-weight rate an observed cohort result still needs repair at the interpretation step. Finish by explaining why none of these fictional summaries determines which college would cause a better outcome for one individual.

Contents · Previous chapter · Next chapter

If a learner averages two group percentages without counts, begin with unequal group sizes. Use five people and 100 people so that the hidden equal-weight assumption becomes visible. Ask them to reconstruct actual outcomes rather than simply supply another formula.

If they can pool correctly but reject a reversal as impossible, return to the shares in X and Y. Give each institution a 100-square paper grid representing its composition. The purpose is to connect the weights with people, rather than decorate a calculation whose meaning remains unclear.

If the mathematics is correct but the conclusion becomes causal, ask what intervention or comparison the records actually establish. Let the learner describe the arithmetic operation in words. The gap is now between a descriptive statistic and a claim about what a university makes happen.

An extension task can allow subgroup advantages to differ and ask which reference mixtures change the result most. A further extension can introduce missing outcomes or rounded rates. Keep the new uncertainty explicit rather than adding complexity without a question.

The lesson follows a practical rhythm: establish the denominator, inspect a worked reversal, compare a common mix, attempt a changed dataset and review the final explanation. Progress appears when the student selects the appropriate measure and preserves its conditions independently. Recognising a familiar paradox label is useful, but explaining a new table is stronger evidence of understanding.

Contents · Previous chapter · Next chapter

Is the overall rate wrong when subgroup rates point the other way?

Not necessarily. Different subgroup proportions can produce the reversal with entirely correct arithmetic. Check counts, definitions and weights before deciding whether the issue is an error or a different comparison question.

Is equal weighting always fair?

No. It is one reference choice. Explain why the chosen weights fit the question, use them consistently for both groups and examine sensitivity when that choice affects the conclusion.

Does standardising the subject mix remove response bias?

Not automatically. Respondents can differ from nonrespondents within subjects. A composition adjustment and a nonresponse assessment address related but distinct problems.

Can I combine published salary medians using cohort sizes?

Those summaries generally do not identify the pooled median. Use the published combined statistic or an appropriate analysis of the underlying distribution. Do not copy a rate-weighting method into a median column.

Does a higher adjusted outcome establish better teaching?

No. It describes the comparison under stated observed rates and weights. Teaching quality and causal effects require additional evidence and a suitable research design.

What should a prospective student do next?

Use the relevant programme’s current official information, examine its required work and support, and ask precise questions about the outcome table. A statistic can guide investigation without becoming a personal guarantee.

Contents · Previous chapter · Next chapter

For institutional learning and responsibility, return to The Gold Standard of Universities and the Higher Education guide. For course exploration, use the International Courses and University Fit Hub.

For survey definitions and practical decisions, read Reading a Graduate Employment Survey, Graduate Tracer Studies and Labour-Market Feedback and Should I Choose a University Course Based on Graduate Salaries?. Those guides provide the broader contexts rather than being replaced by this calculation.

For the underlying statistical relationships, follow How Joint Probability Works for joint and marginal patterns, and How Correlation Works for association, confounding and interpretation. They extend the mathematical route without turning an institutional comparison into a causal conclusion.

Primary source routes reviewed on 21 October 2026 are the ONS explanation of Simpson's paradox and composition and the MOE Graduate Employment Survey dataset and accompanying notes. The institutions, binary outcomes, score sets and all worked calculations here are original fictional examples.

Continue into Education: learning that transfers into life and How X Works. The next useful habit is simple: whenever a comparison surprises you, inspect the people and weights behind the headline before deciding what story the number tells.

Contents · Previous chapter · Return to Universities