Chapter map

1. 1. Event
2. 2. Cost
3. 3. Loss
4. 4. Ratio
5. 5. Forecast rule
6. 6. Hits
7. 7. False alarms
8. 8. Misses
9. 9. Action expense
10. 10. Total expense
11. 11. Climatology
12. 12. Perfect forecast
13. 13. Value
14. 14. User variation
15. 15. Uncertainty
16. 16. Claim

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Event

The useful question is not only how to calculate, but what the calculation assumes. Define one protectable event.

A concrete situation

Twenty of 100 cases occur.

What the mathematics does not license

Changing event changes losses. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Freeze target. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. Cost

A small example makes the mechanism visible. Set protection cost C=20.

A concrete situation

Pay it whenever action is taken.

What the mathematics does not license

Costs may vary in reality. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

State assumption. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Loss

This is where modelling choices begin to affect evidence. Set unprotected event loss L=100.

A concrete situation

Assume full prevention by action.

What the mathematics does not license

Partial protection needs another model. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Name scope. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Ratio

The central mathematical idea is concrete. Compute C/L=.20.

A concrete situation

This is the probability threshold.

What the mathematics does not license

Require 0<C<L. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Show units cancel. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Forecast rule

Here the geometry becomes operational. Act thirty times.

A concrete situation

Fifteen actions coincide with events.

What the mathematics does not license

Use a fixed threshold. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Audit decisions. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Hits

This step deserves a careful pause. Count fifteen protected events.

A concrete situation

Pay cost, avoid loss.

What the mathematics does not license

Do not count hit as free. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Include cost. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. False alarms

The useful question is not only how to calculate, but what the calculation assumes. Count fifteen non-events with action.

A concrete situation

Pay cost without avoided loss.

What the mathematics does not license

False alarm still has consequence. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Retain count. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Misses

A small example makes the mechanism visible. Five events remain unprotected.

A concrete situation

Pay loss 5(100)=500.

What the mathematics does not license

Misses may have heterogeneous losses. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

State simplification. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Action expense

This is where modelling choices begin to affect evidence. Compute 30(20)=600.

A concrete situation

Include hits and false alarms.

What the mathematics does not license

Do not double-count. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Show arithmetic. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Total expense

The central mathematical idea is concrete. Add 600+500.

A concrete situation

Total=1100, mean=11.

What the mathematics does not license

Mean depends on case set. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Report both. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Climatology

Here the geometry becomes operational. At p=.20 and C/L=.20, always and never tie.

A concrete situation

Expense=2000, mean=20.

What the mathematics does not license

Boundary convention should be stated. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Name reference. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Perfect forecast

This step deserves a careful pause. Act only for twenty events.

A concrete situation

Expense=400, mean=4.

What the mathematics does not license

Perfect information is a benchmark. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Avoid attainable claim. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Value

The useful question is not only how to calculate, but what the calculation assumes. Compute (20-11)/(20-4).

A concrete situation

Relative value=.5625.

What the mathematics does not license

Value depends on C/L. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Plot curve. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. User variation

A small example makes the mechanism visible. Repeat across ratios.

A concrete situation

Different users obtain different value.

What the mathematics does not license

One scalar cannot represent everyone. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Show range. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. Uncertainty

This is where modelling choices begin to affect evidence. Resample matched cases and consequences.

A concrete situation

Report sensitivity scenarios.

What the mathematics does not license

Cost estimates may be uncertain. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Separate sources. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Claim

The central mathematical idea is concrete. Name C,L,base rate and protection assumption.

A concrete situation

Teaching value=.5625.

What the mathematics does not license

Do not call value intrinsic to model. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Bound conclusion. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

The cost-loss model asks when a decision maker should pay a protection cost C to avoid a loss L if an event occurs. If protection fully avoids the loss and 0<C<L, the probability threshold is C/L: act when the forecast probability exceeds the cost-loss ratio. Forecast value then depends on both forecast quality and the user’s consequences.

Use one hundred cases with twenty events, protection cost 20 and unprotected event loss 100, so C/L=.20. Suppose a forecast-based rule acts thirty times, containing fifteen hits and fifteen false alarms, and misses five events. Total expense is 30(20)+5(100)=600+500=1100, or 11 per case.

With only climatology p=.20, always act and never act both cost 2000 in this boundary example, or 20 per case. A perfect forecast acts only for the twenty events and costs 20(20)=400, or 4 per case. Relative economic value is (20-11)/(20-4)=9/16=.5625.

The calculation assumes protection is complete, costs are constant and consequences are correctly specified. Different users have different C/L ratios, and value curves should show the range rather than collapse every user into one number. A responsible analysis reports hits, misses, false alarms, base rate, assumptions and uncertainty, and never treats economic value as a universal property of the forecast alone.

---

Frequently asked questions

What is the cost-loss threshold?

C/L when protection fully avoids a loss and 0<C<L.

What is a hit?

Action is taken and the event occurs, so protection cost is paid and loss is avoided.

Why does value vary by user?

Users can have different costs, losses and action capabilities.

What is perfect-forecast expense?

A benchmark that acts only when the event occurs under the assumed model.

Is relative value a universal forecast property?

No. It depends on consequence ratio, base rate, rule and reference.

What must be reported?

Event, C, L, protection effect, counts, base rate, reference, equations, uncertainty and value curve.

---

Primary source and fact-check note

David S. Richardson, “Skill and Relative Economic Value of the ECMWF Ensemble Prediction System,” Quarterly Journal of the Royal Meteorological Society 126(563), 649–667 (2000), DOI 10.1002/qj.49712656313. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• Decision curve analysis
• Brier skill score
• Reliability diagrams
• How Uncertainty Quantification Works
• eduKateSingapore learning routes

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: decision-focused validation and responsible use

Target audit

Freeze event, positive class, score direction, case set, action and consequence model before evaluation.

Formula audit

Write every pair count, threshold count, odds weight, cost, loss, expense, reference and denominator explicitly.

Sample audit

Report positives, negatives, prevalence, event rate, missing cases, weights and dependence.

Threshold audit

Choose operating regions from real decisions before inspecting results; do not optimise on the evaluation outcomes.

Reference audit

Compare model curves with transparent no-skill, treat-all, treat-none, climatology or perfect-information references as appropriate.

Calibration audit

Add probability calibration evidence because ranking and threshold metrics can look favourable with wrong probabilities.

Uncertainty audit

Use paired, dependence-aware uncertainty and sensitivity analysis for scores, curves and consequences.

Final claim

State the exact ranking or conditional-value evidence and stop before universal, causal, salary or guaranteed-outcome claims.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Event

Begin with this mechanism: Define one protectable event. Recreate the concrete case: Twenty of 100 cases occur. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Changing event changes losses. Complete the transfer step: Freeze target. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 2: Cost

Begin with this mechanism: Set protection cost C=20. Recreate the concrete case: Pay it whenever action is taken. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Costs may vary in reality. Complete the transfer step: State assumption. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 3: Loss

Begin with this mechanism: Set unprotected event loss L=100. Recreate the concrete case: Assume full prevention by action. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Partial protection needs another model. Complete the transfer step: Name scope. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 4: Ratio

Begin with this mechanism: Compute C/L=.20. Recreate the concrete case: This is the probability threshold. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Require 0<C<L. Complete the transfer step: Show units cancel. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 5: Forecast rule

Begin with this mechanism: Act thirty times. Recreate the concrete case: Fifteen actions coincide with events. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Use a fixed threshold. Complete the transfer step: Audit decisions. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 6: Hits

Begin with this mechanism: Count fifteen protected events. Recreate the concrete case: Pay cost, avoid loss. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Do not count hit as free. Complete the transfer step: Include cost. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 7: False alarms

Begin with this mechanism: Count fifteen non-events with action. Recreate the concrete case: Pay cost without avoided loss. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: False alarm still has consequence. Complete the transfer step: Retain count. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 8: Misses

Begin with this mechanism: Five events remain unprotected. Recreate the concrete case: Pay loss 5(100)=500. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Misses may have heterogeneous losses. Complete the transfer step: State simplification. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 9: Action expense

Begin with this mechanism: Compute 30(20)=600. Recreate the concrete case: Include hits and false alarms. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Do not double-count. Complete the transfer step: Show arithmetic. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 10: Total expense

Begin with this mechanism: Add 600+500. Recreate the concrete case: Total=1100, mean=11. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Mean depends on case set. Complete the transfer step: Report both. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 11: Climatology

Begin with this mechanism: At p=.20 and C/L=.20, always and never tie. Recreate the concrete case: Expense=2000, mean=20. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Boundary convention should be stated. Complete the transfer step: Name reference. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 12: Perfect forecast

Begin with this mechanism: Act only for twenty events. Recreate the concrete case: Expense=400, mean=4. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Perfect information is a benchmark. Complete the transfer step: Avoid attainable claim. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 13: Value

Begin with this mechanism: Compute (20-11)/(20-4). Recreate the concrete case: Relative value=.5625. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Value depends on C/L. Complete the transfer step: Plot curve. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 14: User variation

Begin with this mechanism: Repeat across ratios. Recreate the concrete case: Different users obtain different value. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: One scalar cannot represent everyone. Complete the transfer step: Show range. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 15: Uncertainty

Begin with this mechanism: Resample matched cases and consequences. Recreate the concrete case: Report sensitivity scenarios. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Cost estimates may be uncertain. Complete the transfer step: Separate sources. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 16: Claim

Begin with this mechanism: Name C,L,base rate and protection assumption. Recreate the concrete case: Teaching value=.5625. Write C, L, C/L, actions, hits, false alarms, misses, expenses, reference and relative value with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Do not call value intrinsic to model. Complete the transfer step: Bound conclusion. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.