WHY MATHEMATICS?

Why Mathematics? | Decision Curve Analysis: Net Benefit, Threshold Probability, Treat-All and Treat-None Strategies

How threshold odds translate prediction counts into net benefit, with a complete hundred-case example, limits and responsible use.

Why mathematics is important becomes easier to see when a real decision depends on it. This guide follows the mechanism from definitions and dimensions to a worked calculation, validation, practical learning and responsible use. It keeps factual evidence separate from interpretation and treats every illustrative scenario as an example rather than a promise of results.

Choose a reading route

Translate threshold probability into an odds weight
Calculate model net benefit from true and false positives
Compare treat-all and treat-none strategies
Keep calibration, validation and consequence assumptions explicit

Chapter map

1. Sample
2. Threshold
3. Actions
4. False positives
5. Odds weight
6. TP term
7. FP penalty
8. Net benefit
9. Treat none
10. Treat all
11. Compare
12. Curve
13. Calibration
14. Uncertainty
15. External validation
16. Claim

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

1. Sample

A small example makes the mechanism visible. Use N=100.

A concrete situation

Twenty events occur.

What the mathematics does not license

Case mix defines baseline. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

State validation set. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

2. Threshold

This is where modelling choices begin to affect evidence. Set p_t=.20.

A concrete situation

This encodes a consequence tradeoff.

What the mathematics does not license

A threshold must come from a real decision. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Justify range. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

3. Actions

The central mathematical idea is concrete. Model flags thirty cases.

A concrete situation

Fifteen are true positives.

What the mathematics does not license

Counts need matched outcomes. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Audit table. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

4. False positives

Here the geometry becomes operational. Thirty minus fifteen gives fifteen.

A concrete situation

Keep FP separate from FN.

What the mathematics does not license

Errors have different weights. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Label cells. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

Did You Know? In this chapter, the most important number is useful only together with its construction rule. Changing scaling, sample membership or validation design can change the numerical answer without changing the arithmetic.

5. Odds weight

This step deserves a careful pause. Compute .20/.80.

A concrete situation

Weight=.25.

What the mathematics does not license

Threshold probability is not raw cost itself. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Explain mapping. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

6. TP term

The useful question is not only how to calculate, but what the calculation assumes. Compute 15/100.

A concrete situation

Contribution=.15.

What the mathematics does not license

This is per evaluated case. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Keep denominator. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

7. FP penalty

A small example makes the mechanism visible. Compute (15/100)(.25).

A concrete situation

Penalty=.0375.

What the mathematics does not license

A different threshold changes weight and counts. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Recalculate together. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

8. Net benefit

This is where modelling choices begin to affect evidence. Subtract .0375 from .15.

A concrete situation

NB=.1125.

What the mathematics does not license

Units are event-equivalent rate under the formula. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Avoid currency claim. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

9. Treat none

The central mathematical idea is concrete. Set TP=FP=0.

A concrete situation

NB=0.

What the mathematics does not license

Zero is a comparator, not automatic safety. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Name policy. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

10. Treat all

Here the geometry becomes operational. Use TP=20,FP=80.

A concrete situation

NB=.20-.80(.25)=0.

What the mathematics does not license

Formula assumes binary action and stated consequences. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Show arithmetic. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

11. Compare

This step deserves a careful pause. Model exceeds both comparators at .20.

A concrete situation

Difference=.1125.

What the mathematics does not license

This is threshold-specific. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Avoid universal claim. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

12. Curve

The useful question is not only how to calculate, but what the calculation assumes. Repeat over defensible thresholds.

A concrete situation

Plot every strategy.

What the mathematics does not license

Data-driven range selection can exaggerate value. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Predeclare. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

13. Calibration

A small example makes the mechanism visible. Check predicted risks before decision analysis.

A concrete situation

Recalibrate if justified on separate data.

What the mathematics does not license

Miscalibration shifts actions. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Add curve. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

14. Uncertainty

This is where modelling choices begin to affect evidence. Use validated intervals or resampling.

A concrete situation

Preserve paired models.

What the mathematics does not license

Small event counts produce unstable curves. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Report events. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

15. External validation

The central mathematical idea is concrete. Evaluate on data separated from fitting.

A concrete situation

Match intended population.

What the mathematics does not license

In-sample net benefit is optimistic. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Keep split. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

16. Claim

Here the geometry becomes operational. Name threshold, outcomes and consequence assumptions.

A concrete situation

Teaching NB=.1125.

What the mathematics does not license

Do not issue individual advice. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Bound conclusion. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

A seven-day practice lab

Day 1: define the decision, observation unit, X variables and Y or reference condition.
Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
Day 3: build a transparent baseline and write what it assumes.
Day 4: fit the main method inside a training-only pipeline; record every parameter.
Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
Day 6: compare validation metrics with the practical workload created by errors.
Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

Which part of the model is a mathematical definition, and which part is a choice?
What small calculation would catch a sign, scale or dimension error?
What evidence comes from training, validation and the untouched test?
What benign condition could look like a fault or prediction failure?
How would the answer change for another response, asset, school project or time period?
What should a human review before acting on the score?

A method-specific audit before deployment

Decision curve analysis connects a predicted probability threshold with the relative weight placed on false positives and false negatives. At threshold p_t, a common net-benefit formula is TP/N minus FP/N multiplied by p_t/(1-p_t). The model curve is compared with simple strategies such as act for nobody and act for everybody across a clinically or operationally defensible threshold range.

Suppose N=100, twenty events occur and a model at threshold .20 flags thirty cases: fifteen true positives and fifteen false positives. The odds weight is .20/.80=.25. Net benefit is 15/100-(15/100)(.25)=.15-.0375=.1125.

Treat-none net benefit is zero. Treat-all gives .20-(.80)(.25)=0 in this teaching sample. The model therefore has higher net benefit at this specific threshold under the assumed consequence ratio. This number is an event-equivalent rate under the chosen formula, not a currency amount, guaranteed benefit or individual recommendation.

Decision curves inherit model calibration, sample composition and threshold assumptions. They should use external or properly separated validation, intervals when useful and a threshold range chosen from real decisions rather than visual convenience. Students can see how odds, opportunity costs and classification counts translate predictions into conditional action comparisons.

Frequently asked questions

What is net benefit?

A threshold-weighted balance of true positives and false positives per evaluated case.

What does threshold probability represent?

A decision point encoding the relative consequence of false-positive and false-negative actions under the model.

Why compare treat all and treat none?

They are simple reference strategies available without model discrimination.

Is net benefit money?

No. The common formula uses event-equivalent units unless further consequences are explicitly monetised.

Does a positive value prove clinical usefulness?

No. Comparison, validation, threshold relevance and assumptions matter.

What must be reported?

Population, event, threshold range, counts, formula, comparators, calibration, validation and uncertainty.

Primary source and fact-check note

Andrew J. Vickers and Elena B. Elkin, “Decision Curve Analysis: A Novel Method for Evaluating Prediction Models,” Medical Decision Making 26(6), 565–574 (2006), DOI 10.1177/0272989X06295361. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: decision-focused validation and responsible use

Target audit

Freeze event, positive class, score direction, case set, action and consequence model before evaluation.

Formula audit

Write every pair count, threshold count, odds weight, cost, loss, expense, reference and denominator explicitly.

Sample audit

Report positives, negatives, prevalence, event rate, missing cases, weights and dependence.

Threshold audit

Choose operating regions from real decisions before inspecting results; do not optimise on the evaluation outcomes.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Reference audit

Compare model curves with transparent no-skill, treat-all, treat-none, climatology or perfect-information references as appropriate.

Calibration audit

Add probability calibration evidence because ranking and threshold metrics can look favourable with wrong probabilities.

Uncertainty audit

Use paired, dependence-aware uncertainty and sensitivity analysis for scores, curves and consequences.

Final claim

State the exact ranking or conditional-value evidence and stop before universal, causal, salary or guaranteed-outcome claims.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Sample

Begin with this mechanism: Use N=100. Recreate the concrete case: Twenty events occur. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Case mix defines baseline. Complete the transfer step: State validation set. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 2: Threshold

Begin with this mechanism: Set p_t=.20. Recreate the concrete case: This encodes a consequence tradeoff. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: A threshold must come from a real decision. Complete the transfer step: Justify range. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 3: Actions

Begin with this mechanism: Model flags thirty cases. Recreate the concrete case: Fifteen are true positives. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Counts need matched outcomes. Complete the transfer step: Audit table. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 4: False positives

Begin with this mechanism: Thirty minus fifteen gives fifteen. Recreate the concrete case: Keep FP separate from FN. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Errors have different weights. Complete the transfer step: Label cells. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Workshop 5: Odds weight

Begin with this mechanism: Compute .20/.80. Recreate the concrete case: Weight=.25. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Threshold probability is not raw cost itself. Complete the transfer step: Explain mapping. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 6: TP term

Begin with this mechanism: Compute 15/100. Recreate the concrete case: Contribution=.15. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: This is per evaluated case. Complete the transfer step: Keep denominator. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 7: FP penalty

Begin with this mechanism: Compute (15/100)(.25). Recreate the concrete case: Penalty=.0375. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: A different threshold changes weight and counts. Complete the transfer step: Recalculate together. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 8: Net benefit

Begin with this mechanism: Subtract .0375 from .15. Recreate the concrete case: NB=.1125. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Units are event-equivalent rate under the formula. Complete the transfer step: Avoid currency claim. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Workshop 9: Treat none

Begin with this mechanism: Set TP=FP=0. Recreate the concrete case: NB=0. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Zero is a comparator, not automatic safety. Complete the transfer step: Name policy. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 10: Treat all

Begin with this mechanism: Use TP=20,FP=80. Recreate the concrete case: NB=.20-.80(.25)=0. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Formula assumes binary action and stated consequences. Complete the transfer step: Show arithmetic. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 11: Compare

Begin with this mechanism: Model exceeds both comparators at .20. Recreate the concrete case: Difference=.1125. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: This is threshold-specific. Complete the transfer step: Avoid universal claim. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 12: Curve

Begin with this mechanism: Repeat over defensible thresholds. Recreate the concrete case: Plot every strategy. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Data-driven range selection can exaggerate value. Complete the transfer step: Predeclare. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Workshop 13: Calibration

Begin with this mechanism: Check predicted risks before decision analysis. Recreate the concrete case: Recalibrate if justified on separate data. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Miscalibration shifts actions. Complete the transfer step: Add curve. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 14: Uncertainty

Begin with this mechanism: Use validated intervals or resampling. Recreate the concrete case: Preserve paired models. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Small event counts produce unstable curves. Complete the transfer step: Report events. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 15: External validation

Begin with this mechanism: Evaluate on data separated from fitting. Recreate the concrete case: Match intended population. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: In-sample net benefit is optimistic. Complete the transfer step: Keep split. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.

Workshop 16: Claim

Begin with this mechanism: Name threshold, outcomes and consequence assumptions. Recreate the concrete case: Teaching NB=.1125. Write threshold probability, odds weight, N, TP, FP, net benefit, comparator strategies and validation with signs, identifiers and assumptions. Calculate the smallest non-trivial example by hand and reproduce it independently. Challenge this limitation: Do not issue individual advice. Complete the transfer step: Bound conclusion. Change exactly one class label, score, tie, threshold, prevalence, ranked case, positive count, AP convention, true positive, false positive, threshold probability, cost, loss, base rate, action effectiveness or validation split. Explain which mathematical object changes, which curve or value moves and which claim must weaken. Keep target, arithmetic, uncertainty and interpretation separate. Explain once to a student and once to a parent. Add a counterexample involving unchanged AUC with bad calibration, high AP under shifted prevalence, optimistic in-sample net benefit or an attractive cost-loss value under unrealistic full protection. State the evidence needed to distinguish explanations. Finish with one supported sentence, one unsupported sentence and one next measurement.