The Gold Standard of Probability | Reinforced Urns and Why Equal Marginals Do Not Make Trials Independent

Imagine a container holding one red counter and one blue counter. Select a counter uniformly at random, record its colour, put it back and add one extra counter of that same colour. Repeat. The returned counter and the additional counter are separate parts of the instruction.

If the first draw is red, the container now contains two red and one blue. The next red probability is 2/3. If the first draw is blue, it contains one red and two blue, so the next red probability is 1/3. Neither branch has probability 1/2 for its next draw.

This is the classical Pólya reinforcement rule. The important mechanism is visible: the outcome changes the composition, and that composition changes the next selection probabilities. A tree diagram should carry the updated counts at every node. Labelling every outgoing red branch 1/2 would describe a different experiment.

The rule assumes that each counter currently in the container is equally likely to be selected. Real physical mixing might fail to achieve that. Our calculations concern the ideal random-selection model, whose assumptions are stated explicitly.

Contents · Next chapter

There are three useful comparison experiments. With replacement alone, return the selected counter and add nothing. With sampling without replacement, leave the selected counter out. With reinforcement, return it and add another of its colour.

Starting from one red and one blue, the first draw is identical across the three models. The second conditional probability differs sharply.

The words “draw randomly” do not select among these rules. Randomness describes how selection occurs; the updating instruction describes how the population changes. Both are needed.

The existing guide to sampling without replacement studies depletion and the position of a first defect. Reinforcement develops the opposite mechanism: a selected colour gains representation. Importing a fixed binomial formula into either changing-composition model can give an attractive but incorrect answer.

Contents · Previous chapter · Next chapter

Write R for red and B for blue. The probability of RR is (1/2)(2/3) = 1/3. The probability of RB is (1/2)(1/3) = 1/6. By colour symmetry, BB has probability 1/3 and BR has probability 1/6.

All four paths total one: 1/3 + 1/6 + 1/6 + 1/3 = 1. The chance of matching colours is therefore 2/3. Two independent fair coin tosses would match with probability 1/2.

Now ask for the second draw alone. Red occurs on RR or BR, so its probability is 1/3 + 1/6 = 1/2. The unconditional second-draw probability is one-half, even though its probability conditional on the first result is either two-thirds or one-third.

This is the precise mental move to repair: an unconditional probability does not become the probability on every conditional branch. The tree combines branches with their appropriate weights. It does not remove the effect of information available along one particular history.

Contents · Previous chapter · Next chapter

Let A mean the first draw is red and D mean the second draw is red. We have P(A) = P(D) = 1/2, but P(A and D) = 1/3. Independence would require the joint probability to equal (1/2)(1/2) = 1/4. It does not.

The same failure appears conditionally: P(D given A) = 2/3 differs from P(D) = 1/2. Knowing the first colour changes the prediction for the second.

An indicator variable makes the relationship numerical. Set X to one for a red first draw and zero otherwise; define Y similarly for the second draw. Their covariance is E(XY) − E(X)E(Y) = 1/3 − 1/4 = 1/12, which is positive.

Positive covariance here means red on one draw is associated with red on the other more often than under independence. It does not mean every pair must match. The mixed paths still occur. Dependence changes a distribution; it does not remove uncertainty or force a deterministic sequence.

Contents · Previous chapter · Next chapter

The probability of RRR is (1/2)(2/3)(3/4) = 1/4. For RRB it is (1/2)(2/3)(1/4) = 1/12. For RBR it is (1/2)(1/3)(2/4) = 1/12. BRR also has probability 1/12.

Those last three sequences all contain two reds and one blue. Their intermediate compositions differ, but their products agree. The classical rule makes sequences with the same colour counts equally probable. This property is called exchangeability.

Exchangeability concerns rearranging the order within a joint distribution. Independence concerns whether knowing other outcomes changes probabilities. They are different properties. Our urn has the former and lacks the latter.

For a checking exercise, write RBB, BRB and BBR. Each has probability 1/12. BBB has probability 1/4. Adding the eight sequence probabilities gives two quarters plus six twelfths, which is one. Enumerating all eight paths provides a stronger check than examining only the most memorable all-red path.

Contents · Previous chapter · Next chapter

Let K count red draws among three selections. Group the paths by K. Under this particular initial composition and reinforcement rule, each possible count has probability 1/4.

Both distributions have mean 1.5 reds. The reinforced version places more probability at the extremes. Its variance is 1.25, compared with 0.75 for the independent version. Equal means therefore preserve very little about runs or tail outcomes.

The chance of at least one red is 1 − P(K = 0). It is 3/4 in the urn and 7/8 under independence. A calculation using only the one-half marginal probability would miss that difference.

Here the outcome whose risk matters determines the useful summary. If a task needs at least one success, clustering successes together can leave more entirely unsuccessful sequences, even while the expected total number of successes stays unchanged.

Contents · Previous chapter · Next chapter

Now begin with two red and two blue counters, still adding one matching counter after every draw. The first red probability remains 1/2. After a red, however, the next red probability becomes 3/5 rather than 2/3.

The probability of two reds is (1/2)(3/5) = 3/10. Subtracting the independent product 1/4 gives covariance 1/20. Dependence is still positive, but weaker than the 1/12 obtained with a one-red, one-blue start.

For three reds the probability is (2/4)(3/5)(4/6) = 1/5. Compare 1/4 for the smaller urn and 1/8 for independent fair trials. The larger starting population reduces the effect of one added counter.

This transfer check shows why an initial fraction is not a complete model description. Both starts are half red, but the same reinforcement step is large relative to two counters and smaller relative to four. Initial counts and the size of each addition carry information that a fraction alone loses.

Contents · Previous chapter · Next chapter

There is another way to produce the same sequence probabilities as the one-red, one-blue urn. First choose a hidden probability p uniformly between zero and one. Within that run, generate independent red-or-blue trials using that fixed p. Choose a new p for each new run.

Conditional on p, the draws are independent. Before p is known, they are dependent because they share it. For example, the unconditional probability of two reds is the average of p² over the unit interval, which is 1/3. The probability of three reds is the average of p³, which is 1/4.

More generally, averaging p to the power k times (1 − p) to the power n − k reproduces the classical urn probability for a specified sequence with k reds. This is a mathematical representation, not evidence that the physical urn secretly chooses p first.

Two mechanisms can agree on observed sequence probabilities. Observing clustered results alone cannot identify which causal story generated them. For a real application, investigate the process and collect evidence that could distinguish explanations. A convenient model should remain a model.

Contents · Previous chapter · Next chapter

For the one-red, one-blue urn, the probability of n consecutive reds telescopes: (1/2)(2/3)…(n/(n + 1)) = 1/(n + 1). At ten draws it is 1/11. Under independent fair trials it would be 1/1024.

These are probabilities assigned before the sequence begins. After ten reds in the reinforced urn, the composition is eleven red and one blue. The next red probability is 11/12. Under a known independent fair-coin model, ten heads would leave the next-head probability at 1/2.

Neither rule should be chosen merely because a striking run occurred. If the real mechanism is uncertain, the run is evidence to assess alongside knowledge of the process. It is not automatic proof of reinforcement.

Likewise, the symmetry that gives each numbered urn draw a one-half red probability does not promise that each realised long sequence will contain exactly half red. Ensemble symmetry and the behaviour of one history answer different questions. Keep that distinction visible before applying familiar long-run slogans.

Contents · Previous chapter · Next chapter

For a paper simulation, record the starting counts, the selected colour, the counts after replacement and the counts after addition. Keep the random-selection method separate from the updating rule. A spreadsheet can calculate the next-red probability from the current red count divided by the current total.

If comparing simulated distributions, restart the urn before each independent run. Ten thousand draws from one growing urn are not ten thousand independently restarted three-draw experiments. The unit of repetition matters.

Useful checks include complete path probabilities summing to one, the expected red count agreeing with symmetry, and a no-addition version reproducing the independent replacement model. A simulation supports debugging; exact two- and three-draw calculations explain why the result occurs.

For actual data, ask whether earlier outcomes change later opportunities, whether observations share a hidden environment, and whether the sampling record preserves order and grouping. These questions can expose a dependence assumption that a single success percentage leaves invisible. They do not, by themselves, estimate a validated model.

Contents · Previous chapter · Next chapter

Check one: an asymmetric start

Begin with one red and two blue counters, adding one matching colour each time. Find the probability of RR. It is (1/3)(2/4) = 1/6. The second red marginal is 1/6 + (2/3)(1/4) = 1/3. Equal marginals again coexist with dependence because 1/6 differs from 1/9.

Check two: a larger reinforcement step

Start with one red and one blue, adding two matching counters after each draw. Following a red, the container has three red and one blue. Thus P(RR) = (1/2)(3/4) = 3/8. Stronger reinforcement increased this joint probability from 1/3.

Check three: repair an incorrect explanation

A student writes: “Every draw is half red, so the chance of three reds is one-eighth.” The error is the unsupported independence step. In the original urn, the branch probabilities are 1/2, then 2/3, then 3/4 along RRR. Their product is 1/4.

Explain each denominator before calculating. Then change the addition rule to zero and show why one-eighth becomes correct for that different experiment. Being able to repair the assumption is the transfer goal.

Contents · Previous chapter · Next chapter

Does replacement guarantee independence?

No. Replacement restores the drawn counter, but an additional updating rule can still change the next distribution. Specify the whole procedure.

Does exchangeable mean order never matters?

It means sequences with rearranged positions have equal joint probabilities under this model. The observed history still changes the conditional prediction through its colour counts. Other reinforcement rules need not be exchangeable.

Can this explain popularity or habits directly?

It can suggest a question about feedback. A real explanatory claim needs evidence about opportunities, observation and the updating mechanism. The urn arithmetic alone supplies none of those empirical facts.

Which reading route fits next?

For the foundations, return to The Gold Standard of Probability, then use conditional probability, joint probability and covariance to separate marginal, conditional and joint questions.

For changing populations and stopping rules, compare sampling without replacement and the first defect with first success, retry limits and unfinished runs. For summaries and long-run assumptions, continue through expected value, variance and the law of large numbers.

For wider learning, The Gold Standard of Education connects explanation with transfer; How X Works opens other subject routes.

The formal model and exchangeability are developed in Kyle Siegrist’s Pólya’s Urn Process. Penn State’s independence lessons support the joint-probability criterion. All small-path comparisons and changed-rule exercises here are worked directly from their stated selection rules.

Contents · Previous chapter · Return to Probability