Find your next learning step
A sequence can look the same after every finite reordering without being independent. Exchangeability expresses that symmetry. De Finetti’s representation explains an infinite exchangeable Bernoulli sequence as a mixture of conditionally independent coin-toss sequences, with a latent success probability shared across observations. The distinction matters: the trials can be independent after conditioning on that probability while remaining positively dependent before it is known.
Open the chapter contents · Probability root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the decision, or read from Chapter 1. The chapters remain open; these grouped routes are a navigable index.
Route 1 · Define the decision
Route 2 · Work the ledgers
Route 3 · Transfer the method
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE DECISION
1. Define exchangeability as a joint symmetry
Random variables X1, …, Xn are exchangeable when every permutation has the same joint distribution. For binary observations, the probability of a particular ordered string depends only on its number of successes, not on where those successes appear.
Exchangeability is weaker than independence. Independent and identically distributed observations are exchangeable, but exchangeable observations can share a hidden condition that makes them dependent.
CHAPTER 2 OF 12 · DEFINE THE DECISION
2. Introduce the latent probability carefully
For an infinite exchangeable Bernoulli sequence, de Finetti’s theorem gives a random parameter Θ in [0,1]. Conditional on Θ=θ, the observations are independent Bernoulli trials with success probability θ.
Unconditionally, the model averages across possible θ values. That mixture creates dependence because an early success makes larger values of Θ more plausible and therefore changes predictions for later observations.
CHAPTER 3 OF 12 · WORK THE LEDGERS
3. Work a two-point mixture
Let Θ equal 0.2 or 0.8 with probability one half each. Given Θ, tosses are independent. Before Θ is observed, each toss still has marginal success probability 0.5.
| Quantity | Calculation | Result |
|---|---|---|
| P(X1=1) | ½(0.2)+½(0.8) | 0.50 |
| P(X1=1,X2=1) | ½(0.2²)+½(0.8²) | 0.34 |
| Product of marginals | 0.50×0.50 | 0.25 |
| Covariance | 0.34−0.25 | 0.09 |
The joint success probability exceeds the product of the marginals. The observations are positively dependent even though each conditional model is an independent sequence.
CHAPTER 4 OF 12 · WORK THE LEDGERS
4. Update the mixture after observations
Suppose the first two observations are successes. The likelihood is 0.2²=0.04 under the low-success component and 0.8²=0.64 under the high-success component.
| Latent Θ | Prior weight | Likelihood of two successes | Posterior weight |
|---|---|---|---|
| 0.2 | 0.50 | 0.04 | 0.04/(0.04+0.64)=0.0588 |
| 0.8 | 0.50 | 0.64 | 0.64/(0.04+0.64)=0.9412 |
The predictive chance of a third success is 0.0588(0.2)+0.9412(0.8)=0.7647. The first two outcomes teach us about the shared Θ; they do not physically cause the third toss.
CHAPTER 5 OF 12 · WORK THE LEDGERS
5. Read counts without discarding order too soon
Within an exchangeable binary block, every sequence with the same number of successes has equal probability. A count can therefore be a sufficient summary for prediction under the model.
That conclusion depends on the exchangeability claim. If time order carries drift, learning, wear or intervention effects, replacing the sequence with a count can erase the mechanism that matters.
CHAPTER 6 OF 12 · TRANSFER THE METHOD
6. Separate infinite and finite representation claims
The exact de Finetti representation applies to an infinitely extendible exchangeable sequence. A finite exchangeable block need not be the first part of any infinite exchangeable process.
Finite de Finetti results instead give approximations under explicit conditions. “The sample is unordered” is not enough to claim an exact mixture of independent Bernoulli trials.
CHAPTER 7 OF 12 · TRANSFER THE METHOD
7. Changed-context transfer: sampling without replacement
Draw two balls without replacement from an urn containing one red and one blue ball. The ordered outcomes red–blue and blue–red are equally likely, so the pair is exchangeable.
Yet the indicators of drawing red are negatively dependent: after a red first draw, a red second draw is impossible. This finite exchangeable pair is a useful warning against treating exchangeability as independence or automatically invoking the infinite theorem.
CHAPTER 8 OF 12 · TRANSFER THE METHOD
8. Connect dependence, urns and repeated events
The existing guide to reinforced urns and equal marginals shows how history can generate dependence. The guide to Borel–Cantelli lemmas and infinitely-often events shows why marginal probabilities alone also fail for long-run occurrence claims.
Use sampling without replacement when the physical mechanism is depletion rather than a latent mixture.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
Exchangeability does not say that observed order was literally randomised, and it is not evidence that the process is stable forever. It is a modelling symmetry that must be defended for the variables and horizon being analysed.
A mixing distribution is not automatically a physical population of coins. It can be a mathematical representation. Likewise, conditional independence holds only after conditioning on the relevant latent quantity and model assumptions.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Practical evidence and checking routine
Define the observational unit, the permitted permutations and the horizon. Compare ordered summaries for drift, batches or interventions before pooling. Calculate at least one joint probability rather than checking marginals alone.
When using a mixture, report the mixing model, posterior predictive calculation and sensitivity to alternative latent structures. For a finite dataset, state whether infinite extendibility is assumed, justified or merely approximated.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Are exchangeable variables identically distributed?
Yes, their one-variable marginals match, but matching marginals do not imply independence.
Why do observations become dependent after mixing?
They share the same uncertain latent parameter, so one observation changes what is believed about the conditions governing the others.
Does de Finetti prove every finite symmetric dataset is IID?
No. The exact representation is an infinite-sequence result; finite versions need extendibility or approximation conditions.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Sources and next reading
The University of Oxford notes on Exchangeability and de Finetti’s Theorem develop the representation and conditional-independence viewpoint. Stanford’s report on finite forms of de Finetti’s theorem records why finite exchangeability requires a qualified statement. The two-point mixture, posterior update and urn transfer are original teaching examples.
Continue through the Probability root, the Gold Standard of Education and How X Works.
Top · Contents · Previous · How X Works