VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

The Core Aim of Education | Why Random Sampling Is Not the Same as Random Assignment

Did you know that a study can use a coin toss and still have a biased sample? The coin may have assigned volunteers fairly to two activities, but that does not explain how the volunteers were chosen. Students regularly hear the word ‘random’ and assume it makes every part of a school project fair. Research asks two different questions before accepting that explanation.

Random sampling is a chance-based way of selecting who enters a sample from a defined population. Random assignment is a chance-based way of deciding which condition each participating unit receives in a comparison or experiment. Sampling concerns who is studied; assignment concerns what happens to those studied. Neither is automatically achieved by simply calling a project ‘random’.

  • Name the population you hope to describe and the actual people or units eligible for selection.
  • Ask whether each unit was sampled by a defined chance method or merely volunteered.
  • Ask separately how selected units were allocated to conditions.
  • Use probability to understand selection and allocation; do not assume random means perfectly balanced.
  • Limit conclusions to what design, sample size, measurements and ethics actually support.

A fully invented club experiment with two separate lotteries

Imagine a fictional school club with 60 eligible members. The club wants to test whether a short worked-example routine helps members solve an unfamiliar puzzle. With appropriate consent, 12 members are selected from the complete list of 60 by a fair random-number draw without replacement. Each has a 12 ÷ 60 = 20% probability of inclusion in that simple design. This first lottery is random sampling from the 60 club members, not from every child in the school or in Singapore.

The twelve selected members are then randomly split into two groups of six using shuffled labels: six labels say A and six say B. Every selected member has a 6 ÷ 12 = 50% chance of being assigned to A in this allocation. Group A practises a puzzle with the worked-example routine; Group B practises a comparable puzzle with another ordinary routine. This second lottery is random assignment. A fair experiment would also try to keep instructions, time, room and measurement comparable. The presence of two lotteries does not remove these other design responsibilities.

Suppose the invented final scores out of ten are A: 7, 8, 8, 9, 6, 10; B: 6, 7, 7, 8, 6, 8. A’s six scores total 48, giving a mean of 48 ÷ 6 = 8 points. B’s total is 42, giving 42 ÷ 6 = 7 points. The observed difference is 1 point on this made-up puzzle measure. These data were selected for teaching arithmetic. They are not evidence that the worked-example routine produces any actual educational benefit.

Even with true random assignment, a small single trial can show chance imbalances and variation. We have not supplied a baseline, uncertainty analysis, replication or evidence that the two puzzle forms have equal difficulty. We may state the descriptive 1-point difference under the imagined conditions; we must not say the method ‘proved’ a one-point improvement for everyone.

What happens if you change just one of the two lotteries?

Scenario A: convenience volunteers, but genuine random assignment

Suppose twelve interested club members respond to a notice and then draw A/B labels at random. The allocation can still be random, but the participants did not form a random sample of all sixty club members. Eager volunteers may differ from quieter members in motivation, available time or experience. Random assignment helps compare routines within those who entered, but it cannot by itself justify claiming that the results represent all club members.

Scenario B: random sample, but students choose their own conditions

Suppose twelve names are drawn fairly from the full list, then students decide whether to join the worked-example routine. That was random sampling but not random assignment. Pupils already confident with puzzles may preferentially choose one condition. Any observed mean difference may reflect initial confidence, prior knowledge or other factors rather than the routine itself.

Scenario C: neither step is random

If a teacher chooses the highest-performing pupils for Group A and the lowest-performing for Group B, then compares their later test marks, neither selection nor assignment is random. The higher score may simply reflect a difference that existed before the lesson. A numerical graph cannot repair a design that lacks comparable groups.

Scenario D: both steps are random

A complete probability sample and random assignment can make a study more interpretable, but they still do not guarantee accurate measurements, adequate sample size, perfect compliance, no dropouts or results that apply forever. We should be especially cautious when reading a single classroom outcome from six pupils per group.

Why these two methods answer different scientific questions

Sampling addresses a population question: to whom might the observations generalise? A properly specified probability sample can help reduce certain selection biases and support statistical inference to its sampling frame. But missing responses, outdated membership lists and nonparticipation can still distort the observed sample. Random selection is a method, not a magical certificate of representativeness.

Assignment addresses a comparison question: if groups differ afterward, how much of the difference can reasonably be attributed to the condition? Chance allocation reduces systematic control over group membership and helps make competing causes less likely to align deliberately with treatment. However, no single random draw can guarantee that two very small groups start identical; analysts must consider uncertainty and the actual design.

The US Office of Research Integrity’s educational guide to random assignment explains allocation by chance in experiments. For a school project, the most important transfer is conceptual: never substitute a chance-based method for another one just because both contain the word ‘random’.

How to recognise misleading wording in a project report

Consider these statements: ‘We randomly chose six volunteers for each programme’; ‘We sampled randomly from the class list’; ‘We flipped a coin to decide the activities’; and ‘We asked whoever was free’. Which one tells you about sampling, and which one about assignment? The first is ambiguous: ‘volunteers’ may mean convenience recruitment, while the allocation may be random. The second discusses sample selection. The third describes assignment. The fourth is convenience selection.

A stronger report makes both steps explicit: ‘We randomly selected twelve members from the club’s complete sixty-member list, and then randomly assigned six to each condition.’ It should add the dates, consent arrangements, excluded cases, task definition and how outcomes were measured. The details are not bureaucracy; they let another reader judge what conclusions can be defended.

The two-minute classroom diagnostic

Ask: ‘Ten students volunteered for a trial. Their names were put in a hat. Five drawn names joined a new practice method; the other five used the usual method. Was there random sampling from the whole year level?’ Answer: not on the described facts. The students volunteered first, so any chance draw was within the volunteer group. Ask a second question: ‘Was there random assignment to practice conditions?’ If the hat genuinely allocated volunteers to the two conditions, yes.

Now ask the pupil to invent one way the report could make a wrong claim. A clear response might be: ‘The result does not necessarily apply to students who would not volunteer.’ A more advanced response notes that equal numbers alone cannot make the small groups identical in previous skill.

Progressive learning activity

Guided: two labelled boxes

Draw a box labelled ‘eligible population’ and a smaller box labelled ‘selected participants’. Place an arrow called ‘chance selection’ between them. Inside the selected box, draw two groups connected by an arrow called ‘chance allocation’. Let the teacher point to each arrow and ask its purpose. The distinction becomes visible before discussing formulas.

Supported: keep the numbers but change the recruitment rule

Use forty imaginary numbered tokens in a bag. Draw eight tokens without replacement. Each token has 8/40 = 20% selection probability. Shuffle four red and four blue assignment labels among those eight tokens; each selected token has 4/8 = 50% assignment probability to red. Then repeat the allocation while changing how the eight tokens were chosen: the first eight handed to you are not a random sample even if the red–blue split is random.

Independent: diagnose an unfamiliar study description

Present a fresh scenario: ‘A teacher surveys 18 pupils who stayed for a voluntary after-school event; a random-number generator assigns half to read a short article and half to watch a video.’ Without help, learners should state that sampling from the school’s whole pupil population was not random, while assignment within the 18 participants can be random if performed as described. They should also name one limitation before stating any advantage of the design.

Transfer: from experiments to survey decisions

For a school transport survey, choosing pupils randomly from an eligible register is a sampling question. There may be no treatment to assign at all. For comparing two existing classes taught by different methods, neither class assignment nor pupil selection necessarily used random procedures. The existence of two groups does not itself make a controlled randomised experiment.

How parents and teachers can check progress

A learner has moved beyond vocabulary memorisation when they can write a sampling sentence and an assignment sentence for a new study, calculate simple selection probabilities under specified assumptions, and identify which population the sample could speak about. They should also recognise when no causal comparison is justified. Repeating the definition alone is not enough.

Parents can ask ‘Who was allowed into the study?’ before ‘Which group did better?’ Teachers can ask pupils to annotate a methodology paragraph with S for sample selection and A for assignment, then produce one restrained conclusion. If an answer changes when a single recruitment detail changes, that reveals the reasoning is doing useful work.

For school activities involving children, follow the institution’s rules on consent, privacy, safety and fairness. The thought experiments here do not authorise an actual study, intervention or collection of pupil-level data. A classroom can practise the logic with counters, fictional scores and an imaginary club rather than real students.

Frequently asked questions

Does random assignment guarantee identical groups?

No. It removes deliberate assignment preference and supports fairer comparison in expectation, but chance imbalances can occur, especially in small samples. Good reporting does not hide those limitations.

Can a project have random assignment without random sampling?

Yes. Many experiments use a convenience or volunteer sample and randomise those participants into conditions. This helps one kind of inference but does not automatically make the sample representative of a broader population.

Is selecting the first ten names on a class list random?

Not simply because the result seems arbitrary. Alphabetical, arrival or seat order can correlate with other patterns. Describe a genuine chance selection method and the eligible population.

Is a one-point mean difference proof of a useful intervention?

No. It is a descriptive result for the observed scores. Interpretation needs a well-designed comparison, the uncertainty of sampling and assignment, relevant baseline information, task comparability and, where suitable, replication.

Closing: ask two questions before believing one result

Before accepting a project conclusion, ask who had a chance to enter the study and who had a chance to receive each condition. Those questions protect different parts of the reasoning. The best classroom outcome is a student who can explain the difference in a fresh example and qualify a claim without being prompted.

Continue with the eduKateSG Science Learning Hub, the Education Hub and the already published Core Aim guide to comparing treatment and control when both groups improve. The immutable Clementi teaching reference explains how diagnosis, careful examples and independently checked transfer fit together. All club details and scores above are invented.