Find your next learning step
Two signals can both be accurate yet differ in what decisions they support. Blackwell ordering asks a demanding question: can one experiment reproduce the other by adding state-independent noise? If so, every decision maker can do at least as well with the ungarbled experiment. This turns “more information” from a slogan into a testable channel relation, while leaving room for experiments that are genuinely incomparable.
Open the chapter contents · Probability root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the decision, or read from Chapter 1. The chapters remain open; these grouped routes are a navigable index.
Route 1 · Define the decision
Route 2 · Work the ledgers
Route 3 · Transfer the method
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE DECISION
1. Define states, signals and actions before ranking
An experiment is a channel from an unknown state to an observed signal. Write κ(s|θ) for the probability of signal s in state θ. A decision rule then maps signals to actions, and a payoff function says what each state–action pair is worth.
The same channel can be useful for one task and nearly useless for another. Blackwell comparison avoids choosing a favourite payoff in advance: it asks whether one experiment gives weakly greater optimal expected payoff for every finite decision problem and prior.
CHAPTER 2 OF 12 · DEFINE THE DECISION
2. Recognise a garbling kernel
Experiment λ is a garbling of κ when there is a stochastic kernel G, independent of the state once κ’s signal is known, such that λ=κG. Operationally, observe κ, then randomly relabel or blur its output using G. No new state information enters after the first signal.
The order points from the more informative channel to the channel it can simulate. Equality is possible up to reversible garblings. The relation is a partial order after equivalent experiments are identified, so not every pair can be ranked.
CHAPTER 3 OF 12 · WORK THE LEDGERS
3. Work a binary garbling ledger
Let the state be Rain or Dry with equal prior probability. Experiment κ reports the correct state with probability 0.80. A second device flips κ’s reported label with probability 0.25, independently of the true state. The resulting experiment λ is correct with probability 0.80×0.75+0.20×0.25=0.65.
| True state | κ: Rain signal | κ: Dry signal | λ: Rain signal | λ: Dry signal |
|---|---|---|---|---|
| Rain | 0.80 | 0.20 | 0.65 | 0.35 |
| Dry | 0.20 | 0.80 | 0.35 | 0.65 |
| Garbling step | Keep 0.75 | Flip 0.25 | κG | κG |
CHAPTER 4 OF 12 · WORK THE LEDGERS
4. Translate informativeness into decision value
Suppose the action is Carry or Leave and the payoff is 1 for matching Rain with Carry or Dry with Leave, and 0 otherwise. Following κ gives expected payoff 0.80; following λ gives 0.65. The numerical loss is task-specific, but the direction is not: κ can imitate any λ-based rule by first applying G.
Blackwell’s theorem gives the converse in the finite setting. If κ has weakly greater optimal value for every decision problem, then λ can be obtained from κ by a garbling kernel. A single successful task does not establish that universal claim.
CHAPTER 5 OF 12 · WORK THE LEDGERS
5. Change the loss and expose the quantifier
Now make an unnecessary umbrella cost 1 and a missed rain cost 4. With equal priors, the optimal threshold is no longer one half. The value gap changes because signals matter through the action boundary, not through accuracy alone.
| Comparison | What is held fixed | What may change | What it proves |
|---|---|---|---|
| One accuracy score | Prior and classification loss | Nothing | Task-specific ranking |
| One asymmetric loss | Prior and loss table | Optimal action rule | Value for that task |
| Every prior and payoff | State and signal channels | Decision rule | Blackwell dominance |
| Found kernel G | κ and λ | Post-processing only | Constructive dominance |
CHAPTER 6 OF 12 · TRANSFER THE METHOD
6. Solve the matrix test without reversing it
For finite channels, place states in rows and signals in columns. Seek a row-stochastic matrix G with nonnegative entries and rows summing to one such that λ=κG. This is a linear feasibility problem.
If a feasible G exists, preserve it as evidence rather than reporting only a solver verdict. If none exists, that does not prove λ dominates κ; the channels may be incomparable. Test the reverse equation separately.
CHAPTER 7 OF 12 · TRANSFER THE METHOD
7. Connect the established probability routes
The forecast-encompassing guide studies information refinement in calibrated predictions. The conditional-calibration guide explains what a forecast knows relative to an information set.
Use the proper-scoring guide when the comparison is tied to a declared predictive loss, and the posterior-risk guide for squared-error value.
CHAPTER 8 OF 12 · TRANSFER THE METHOD
8. Distinguish Blackwell order from information summaries
Mutual information, accuracy, variance of posteriors and one proper score each compress a channel into a number. Blackwell order preserves all decision values. A Blackwell-dominating channel cannot have less value in a chosen task, but a larger scalar summary need not supply a garbling relation.
The prior also matters to scalar summaries. The channel equation does not depend on a prior, whereas a Bayes risk calculation does. Keep these levels separate when moving from an experiment design to an operational decision.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Know the limits and common misconceptions
Garbling is not any degradation observed in a data pipeline. The post-processing kernel must not secretly depend on the state. Selection bias, missingness driven by the hidden state or a changed population can break the factorisation.
Failure to find a kernel with noisy estimated matrices is not automatically structural incomparability. Sampling error, rounding, unobserved outcomes and mismatched state definitions can make an exact feasibility problem misleading.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Use an evidence and source routine
Record states, signal alphabets, channel orientation, priors used for examples, payoff tables, optimal rules, the candidate kernel, feasibility tolerances, sampling uncertainty and both comparison directions. Preserve enough digits to reproduce λ=κG.
Blackwell’s 1953 paper establishes equivalent comparisons of finite experiments. A modern coarse-graining and Blackwell-order study shows why mutual-information-style capability and Blackwell dominance should not be conflated.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Is the more accurate signal always Blackwell better?
No. Accuracy uses one label decision and one loss. Another channel may reveal information useful for a different action.
Can the kernel be deterministic?
Sometimes, but Blackwell garblings allow random relabelling; restricting to deterministic maps is a stronger condition.
What if each experiment wins one task?
Then neither can dominate the other in Blackwell’s universal decision-value order.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Continue with a kernel transfer check
Change the label-flip probability from 0.25 to 0.10. Recompute λ’s correct-report probability and its payoff in the symmetric matching task. Then suppose Rain reports are flipped with probability 0.10 but Dry reports with probability depending on the hidden state. Explain why that hidden dependence is not a valid post-signal garbling kernel.
Continue through the Probability root, the Gold Standard of Education and the How X Works hub.
Top · Contents · Previous · How X Works