Chapter map

1. 1. Ensemble role
2. 2. Observation
3. 3. First term
4. 4. Pair matrix
5. 5. Self-pairs
6. 6. Ordinary spread
7. 7. Ordinary score
8. 8. Fair denominator
9. 9. Fair spread
10. 10. Fair score
11. 11. Size comparison
12. 12. Exchangeability
13. 13. Missing members
14. 14. Calibration
15. 15. Uncertainty
16. 16. Claim

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Ensemble role

The central mathematical idea is concrete. Treat members as draws when fairness is intended.

A concrete situation

Use 8,10,14.

What the mathematics does not license

Designed scenarios may not be exchangeable. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

State interpretation. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. Observation

Here the geometry becomes operational. Set y=12.

A concrete situation

Member distances are 4,2,2.

What the mathematics does not license

Observation matching errors contaminate score. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Verify time. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. First term

This step deserves a careful pause. Average member distances.

A concrete situation

Result is 8/3.

What the mathematics does not license

Unequal weights need another formula. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Show weights. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Pair matrix

The useful question is not only how to calculate, but what the calculation assumes. Calculate all ordered distances.

A concrete situation

Total is 24.

What the mathematics does not license

Do not omit symmetric entries under this convention. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Audit matrix. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Self-pairs

A small example makes the mechanism visible. Keep three diagonal zeros in ordinary CRPS.

A concrete situation

There are M squared pairs.

What the mathematics does not license

Self-pairs shrink estimated spread. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Explain bias. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Ordinary spread

This is where modelling choices begin to affect evidence. Divide 24 by 2M squared.

A concrete situation

Term is 4/3.

What the mathematics does not license

Denominator defines empirical CRPS. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Label form. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Ordinary score

The central mathematical idea is concrete. Subtract spread from first term.

A concrete situation

CRPS is 4/3.

What the mathematics does not license

Lower is better. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

State orientation. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Fair denominator

Here the geometry becomes operational. Use 2M(M-1).

A concrete situation

Denominator is 12.

What the mathematics does not license

M must exceed one. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Check size. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Fair spread

This step deserves a careful pause. Divide 24 by 12.

A concrete situation

Term is 2.

What the mathematics does not license

This estimates between-draw spread without self-pairs. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Explain expectation. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Fair score

The useful question is not only how to calculate, but what the calculation assumes. Subtract 2 from 8/3.

A concrete situation

Result is 2/3.

What the mathematics does not license

Individual fair scores can be negative. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Do not flag automatically. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Size comparison

A small example makes the mechanism visible. Repeat at different M under same predictive law.

A concrete situation

Fairness targets expected comparability.

What the mathematics does not license

One realisation may still vary. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Use many cases. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Exchangeability

This is where modelling choices begin to affect evidence. Check members have common predictive status.

A concrete situation

Document perturbations and weights.

What the mathematics does not license

Control member plus perturbations may violate symmetry. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Justify. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Missing members

The central mathematical idea is concrete. Predefine incomplete-ensemble handling.

A concrete situation

Record actual M per case.

What the mathematics does not license

Variable M can bias ordinary means. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Audit availability. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Calibration

Here the geometry becomes operational. Use rank histograms and coverage.

A concrete situation

Pair score with distributional diagnostics.

What the mathematics does not license

Fair CRPS does not guarantee calibration. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Add checks. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. Uncertainty

This step deserves a careful pause. Respect serial and spatial dependence.

A concrete situation

Compare paired score differences.

What the mathematics does not license

Naive standard errors can be too small. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

State resampling. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Claim

The useful question is not only how to calculate, but what the calculation assumes. Name ordinary or fair form and member interpretation.

A concrete situation

Teaching scores are 4/3 and 2/3.

What the mathematics does not license

Do not claim correction creates better forecasts. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Bound conclusion. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

An ensemble forecast is often a finite sample from an intended predictive distribution. If the ordinary empirical CRPS is used directly, its expected value depends on ensemble size because the pairwise-spread term includes self-pairs and uses M squared in the denominator. Ferro, Richardson and Weigel study ensemble-size effects and fair adjustments. The fair CRPS replaces the spread denominator with M(M-1), removing the finite-member bias under the exchangeable-sample interpretation.

For members 8,10,14 and observation 12, the mean member-to-observation distance is (4+2+2)/3=8/3. The ordered pairwise distance matrix has total 24. Ordinary empirical CRPS subtracts 24/(2×3 squared)=4/3, so the score is 4/3. Fair CRPS subtracts 24/(2×3×2)=2, so it is 8/3-2=2/3. Lower is better in this convention. The difference is the ensemble-size correction, not new evidence about the observation.

Fair CRPS is appropriate when members can be treated as exchangeable draws from an underlying predictive distribution. It is undefined for a one-member ensemble because M-1 is zero. It can also become negative for an individual case, which is not an error: fairness concerns expected comparison across ensembles, not a requirement that each realised score be nonnegative. If members are deliberately chosen scenarios rather than random draws, the empirical-distribution interpretation may be the relevant one instead.

Report ensemble size, member weights, exchangeability rationale, duplicate or missing members, observation matching, fair or ordinary formula and uncertainty. Compare systems at the same M or use the fair form with its assumptions, and retain rank histograms or calibration diagnostics. Students can see how denominators, self-pairs and sampling interpretation change a score even when the visible ensemble values stay the same.

---

Frequently asked questions

What makes CRPS fair?

The spread term excludes self-pairs through M(M-1) when members are exchangeable draws.

Why does ordinary empirical CRPS depend on M?

Its pairwise expectation includes M zero self-distances.

Can a fair score be negative?

Yes for an individual case; fairness is an expectation property.

What happens at M=1?

The fair denominator is zero, so the formula is undefined.

Are all ensembles exchangeable?

No. Control members, weighted scenarios or structured perturbations need careful interpretation.

What must be reported?

M, member weights, exchangeability, formula, missing members, matched cases, calibration and uncertainty.

---

Primary source and fact-check note

Christopher A. T. Ferro, David S. Richardson and Andreas P. Weigel, “On the Effect of Ensemble Size on the Discrete and Continuous Ranked Probability Scores,” Meteorological Applications 15(1), 19–24 (2008), DOI 10.1002/met.45. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• Weighted interval score
• Winkler interval score
• Quantile regression averaging
• How Uncertainty Quantification Works
• eduKateSingapore learning routes

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: score-specific validation and responsible use

Target audit

Freeze the predictive object, outcome vector or quantile curve, case set, units, horizons and preferred score direction before seeing comparative results.

Formula audit

Write every expectation, distance, indicator, quantile score, exponent, weight and denominator explicitly; identify ordered versus unordered pair conventions.

Computation audit

Reproduce the teaching arithmetic independently and test the implementation on identical members, perfect forecasts, constant vectors and one changed coordinate.

Weight audit

Predeclare percentile weights, coordinate scaling, pair weights or member probabilities; explain their decision meaning and units.

Support audit

Inspect quantile crossing, sparse tails, ensemble size, missing members, missing vector coordinates and weakly sampled pair expectations.

Companion audit

Report an unweighted or marginal score and calibration evidence so regional, geometric or dependence gains cannot hide deterioration elsewhere.

Uncertainty audit

Use paired comparisons and resampling that respect serial, spatial, horizon and forecast-origin dependence.

Final claim

Describe exactly which forecast feature the score evaluates; never turn one total into proof that the whole predictive distribution is correct.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Ensemble role

Begin with this mechanism: Treat members as draws when fairness is intended. Recreate the concrete case: Use 8,10,14. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Designed scenarios may not be exchangeable. Complete the transfer step: State interpretation. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 2: Observation

Begin with this mechanism: Set y=12. Recreate the concrete case: Member distances are 4,2,2. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Observation matching errors contaminate score. Complete the transfer step: Verify time. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 3: First term

Begin with this mechanism: Average member distances. Recreate the concrete case: Result is 8/3. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Unequal weights need another formula. Complete the transfer step: Show weights. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 4: Pair matrix

Begin with this mechanism: Calculate all ordered distances. Recreate the concrete case: Total is 24. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Do not omit symmetric entries under this convention. Complete the transfer step: Audit matrix. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 5: Self-pairs

Begin with this mechanism: Keep three diagonal zeros in ordinary CRPS. Recreate the concrete case: There are M squared pairs. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Self-pairs shrink estimated spread. Complete the transfer step: Explain bias. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 6: Ordinary spread

Begin with this mechanism: Divide 24 by 2M squared. Recreate the concrete case: Term is 4/3. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Denominator defines empirical CRPS. Complete the transfer step: Label form. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 7: Ordinary score

Begin with this mechanism: Subtract spread from first term. Recreate the concrete case: CRPS is 4/3. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Lower is better. Complete the transfer step: State orientation. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 8: Fair denominator

Begin with this mechanism: Use 2M(M-1). Recreate the concrete case: Denominator is 12. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: M must exceed one. Complete the transfer step: Check size. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 9: Fair spread

Begin with this mechanism: Divide 24 by 12. Recreate the concrete case: Term is 2. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: This estimates between-draw spread without self-pairs. Complete the transfer step: Explain expectation. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 10: Fair score

Begin with this mechanism: Subtract 2 from 8/3. Recreate the concrete case: Result is 2/3. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Individual fair scores can be negative. Complete the transfer step: Do not flag automatically. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 11: Size comparison

Begin with this mechanism: Repeat at different M under same predictive law. Recreate the concrete case: Fairness targets expected comparability. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: One realisation may still vary. Complete the transfer step: Use many cases. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 12: Exchangeability

Begin with this mechanism: Check members have common predictive status. Recreate the concrete case: Document perturbations and weights. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Control member plus perturbations may violate symmetry. Complete the transfer step: Justify. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 13: Missing members

Begin with this mechanism: Predefine incomplete-ensemble handling. Recreate the concrete case: Record actual M per case. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Variable M can bias ordinary means. Complete the transfer step: Audit availability. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 14: Calibration

Begin with this mechanism: Use rank histograms and coverage. Recreate the concrete case: Pair score with distributional diagnostics. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Fair CRPS does not guarantee calibration. Complete the transfer step: Add checks. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 15: Uncertainty

Begin with this mechanism: Respect serial and spatial dependence. Recreate the concrete case: Compare paired score differences. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Naive standard errors can be too small. Complete the transfer step: State resampling. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.

Workshop 16: Claim

Begin with this mechanism: Name ordinary or fair form and member interpretation. Recreate the concrete case: Teaching scores are 4/3 and 2/3. Write response units, ensemble members, M, observation distances, ordered pair distances, denominators, ordinary CRPS and fair CRPS with units, signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Do not claim correction creates better forecasts. Complete the transfer step: Bound conclusion. Change exactly one tau level, quantile, quadrature weight, ensemble member, ensemble size, member probability, vector coordinate, scale factor, norm, p order, pair weight, threshold, forecast case or missing-data rule. Explain which mathematical object changes, which score term moves, which ranking survives and which claim must weaken. Keep target definition, calculation, aggregation, uncertainty and interpretation on separate notebook lines. Explain the result once to a student with the full arithmetic and once to a parent with the real decision and original units. Add a counterexample involving quantile crossing, post-result weighting, unequal ensemble sizes, nonexchangeable members, scale domination, common bias, identical marginals with different dependence or a good regional score with poor global performance. State the extra evidence that would separate the explanations. Finish with one supported sentence, one unsupported sentence and one useful next measurement.