Why Mathematics? | E50 and E90 Calibration Errors for Reliability Survival Models: Median, Tail Quantiles, Smooth Curves and Validation

An average can hide whether error is broadly modest or concentrated in a smaller group. Two models can share ICI .04 with different tail behaviour. This is the central mathematical mechanism in why one average is incomplete. Write the quantities and assumptions before using software, then reproduce the calculation or geometric argument in a small example. A summary cannot locate the affected risk region. Inspect the full curve. The useful habit is to keep the physical question, the probability model and the numerical method in separate columns so that a correct computation is not mistaken for complete evidence.

Did You Know? A result can be numerically stable and still answer the wrong question if the failure event, denominator or transformation changes. Test one nearby case and predict the direction of change before recalculating. Save inputs, intermediate values and units so a second reader can audit the path.

For practice, turn the example into a sensitivity check. Change one defensible input while holding the others fixed, compare the new result with the original and explain why it moved. Then construct a counterexample in which the shortcut fails. End with two sentences: what the mathematics supports, and what remains uncertain.

Contents · Next chapter

Start with the operational question behind absolute curve errors. Start with each prediction’s absolute distance from the smooth observed calibration curve at the horizon. In the illustrative case, Predicted .8 and curve .73 give error .07. Draw the relevant graph, density, boundary or timeline before calculating; the picture should use the same variables and sign convention as the equations. These are not binary Brier residuals. Preserve probability units.

A student should be able to rebuild the result without hidden settings. Record the sampling rule or optimisation criterion, the exact failure definition, and the evidence used for every distribution or parameter. Compare a transparent baseline with the advanced method. If they disagree, investigate the mechanism before deciding which answer is more credible.

This chapter belongs to a larger decision, so translate the number carefully. State one action the evidence permits and one action it does not. A parent or educator can ask which school ideas are doing the work: algebra organises relationships, probability represents uncertainty, calculus describes local change, and computing repeats the documented rules at scale.

Contents · Previous chapter · Next chapter

Treat the e50 definition as a model-checking problem rather than a vocabulary exercise. E50 is the median of individual absolute calibration errors. Half the errors lie at or below the median under a continuous distribution. Recompute the example from the raw values and label every unit. Ties and finite samples need a convention. Name the quantile rule.

The strongest check uses evidence the model did not choose for itself. Hold out later observations, an independent simulation set or a known-answer benchmark. Report errors where decisions are sensitive, especially near the failure boundary, instead of relying on one global accuracy score. A method that performs well on easy safe cases can still fail at the rare cases that motivated the analysis.

Now change one assumption and make a directional prediction before running the calculation. If the result moves the opposite way, inspect the sign convention, transformation, density ratio or constraint. This prediction-first routine develops transferable problem-solving skills because it connects symbolic work to a testable expectation.

Contents · Previous chapter · Next chapter

The mathematics of the e90 definition becomes useful only when its mechanism is explicit. E90 is the 90th percentile of individual absolute calibration errors. Consider the worked illustration: About ninety percent of errors are no larger than this population quantile. Preserve the original inputs, calculate step by step and compare with a second method whenever an analytic or high-budget reference exists. It is not necessarily the maximum. Report Emax separately if needed.

There are three uncertainties to keep apart. Sampling or numerical uncertainty comes from finite computation. Parameter uncertainty comes from estimating inputs. Model-form uncertainty comes from simplified physics, distributions or dependence. Reducing the first does not automatically reduce the other two. A credible report names all three and shows which one drives the decision.

For a practical investigation, create a small reproducible notebook or spreadsheet with a frozen data sheet, formula sheet, diagnostics sheet and decision note. Another student should be able to change one input and obtain the same update without guessing what the model meant.

Primary source: https://doi.org/10.1002/sim.8570. Read it for the method and its assumptions; do not transfer its performance claims automatically to another model.

Contents · Previous chapter · Next chapter

Use a ten-error dataset to practise sensitivity analysis. Use .01,.02,.03,.04,.05,.06,.07,.08,.09,.10 as ordered teaching errors. The set makes the order statistics visible. Before accepting the result, identify the quantity that would change the conclusion most. Real errors are not evenly spaced. Keep it illustrative.

A simple baseline is essential. It reveals whether the advanced method is improving accuracy or only adding complexity. Compare model calls, wall time, uncertainty and failure-mode coverage under one declared budget. Repeat stochastic calculations with several seeds; a favourable single run is not evidence of dependable efficiency.

Students can extend the example by adding correlation, a second failure mode or a modest measurement error. Explain which part of the formula changes and which remains fixed. This exercise shows how mathematics transfers from a clean textbook case to engineering and computing without pretending that the classroom model is a certified design tool.

Contents · Previous chapter · Next chapter

Build a small counterexample for calculating e50. For ten ordered values, the common median averages the fifth and sixth values. The initial example is: (.05+.06)/2=.055. Then create a nearby case where the usual shortcut becomes unreliable. Some weighted definitions differ. State the calculation.

Counterexamples are positive learning tools. They reveal the conditions that make a method work and help students recognise when a software output deserves a pause. Keep the counterexample small enough to plot or enumerate. Check sign, support, bounds and conservation of probability before adding complexity.

When writing the conclusion, separate a documented fact from an interpretation and from an illustrative scenario. Cite the primary method paper for the factual mechanism, use the local calculation for the illustration, and label any recommendation as conditional on the stated model. This keeps career and engineering advice age-appropriate and honest.

Contents · Previous chapter · Next chapter

Connect calculating e90 to a real decision while protecting the boundary between calculation and judgement. Under the nearest-rank rule, rank ceiling(.90 times 10)=9 gives .09. The ninth ordered error is .09. First verify the arithmetic, then translate it into an allowed action and an action the evidence does not support. Interpolated software may return another value. Record the quantile type.

The audit should include the original question, input provenance, failure convention, algorithm settings, convergence or stopping rule, and a sensitivity result. For an adaptive method, also preserve the sequence of choices. For an optimisation, preserve every constraint. For Bayesian updating, preserve the prior and observation model. These records turn a result into something another person can challenge and improve.

A final transfer question is useful: where else would the same mathematical idea appear? Density ratios appear in survey correction and machine learning, probabilistic constraints in planning, and conditional probability in medical testing. The context changes, but transparent assumptions and reproducible checks remain valuable.

Contents · Previous chapter · Next chapter

Sample quantiles have several legitimate interpolation definitions, especially in small samples. One package may interpolate between .09 and .10. This is the central mathematical mechanism in why quantile conventions matter. Write the quantities and assumptions before using software, then reproduce the calculation or geometric argument in a small example. Silent defaults hinder reproducibility. Save software and settings. The useful habit is to keep the physical question, the probability model and the numerical method in separate columns so that a correct computation is not mistaken for complete evidence.

Did You Know? A result can be numerically stable and still answer the wrong question if the failure event, denominator or transformation changes. Test one nearby case and predict the direction of change before recalculating. Save inputs, intermediate values and units so a second reader can audit the path.

For practice, turn the example into a sensitivity check. Change one defensible input while holding the others fixed, compare the new result with the original and explain why it moved. Then construct a counterexample in which the shortcut fails. End with two sentences: what the mathematics supports, and what remains uncertain.

Contents · Previous chapter · Next chapter

Start with the operational question behind interpreting e50. E50 describes the typical absolute probability-scale calibration error in the validation distribution. In the illustrative case, E50 .03 means median error of three probability points. Draw the relevant graph, density, boundary or timeline before calculating; the picture should use the same variables and sign convention as the equations. It does not describe every unit. Pair it with E90.

A student should be able to rebuild the result without hidden settings. Record the sampling rule or optimisation criterion, the exact failure definition, and the evidence used for every distribution or parameter. Compare a transparent baseline with the advanced method. If they disagree, investigate the mechanism before deciding which answer is more credible.

This chapter belongs to a larger decision, so translate the number carefully. State one action the evidence permits and one action it does not. A parent or educator can ask which school ideas are doing the work: algebra organises relationships, probability represents uncertainty, calculus describes local change, and computing repeats the documented rules at scale.

Contents · Previous chapter · Next chapter

Treat interpreting e90 as a model-checking problem rather than a vocabulary exercise. E90 highlights the upper part of calibration error without relying on a single extreme maximum. E90 .09 indicates a substantial high-error region. Recompute the example from the raw values and label every unit. It does not say which predictions are affected. Map errors back to risk.

The strongest check uses evidence the model did not choose for itself. Hold out later observations, an independent simulation set or a known-answer benchmark. Report errors where decisions are sensitive, especially near the failure boundary, instead of relying on one global accuracy score. A method that performs well on easy safe cases can still fail at the rare cases that motivated the analysis.

Now change one assumption and make a directional prediction before running the calculation. If the result moves the opposite way, inspect the sign convention, transformation, density ratio or constraint. This prediction-first routine develops transferable problem-solving skills because it connects symbolic work to a testable expectation.

Contents · Previous chapter · Next chapter

The mathematics of choosing the horizon becomes useful only when its mechanism is explicit. Both metrics are defined at a stated prediction time. Consider the worked illustration: Compute at 500 hours for a 500-hour decision. Preserve the original inputs, calculate step by step and compare with a second method whenever an analytic or high-budget reference exists. Values at 200 and 1,000 hours answer different questions. Pre-specify horizons.

There are three uncertainties to keep apart. Sampling or numerical uncertainty comes from finite computation. Parameter uncertainty comes from estimating inputs. Model-form uncertainty comes from simplified physics, distributions or dependence. Reducing the first does not automatically reduce the other two. A credible report names all three and shows which one drives the decision.

For a practical investigation, create a small reproducible notebook or spreadsheet with a frozen data sheet, formula sheet, diagnostics sheet and decision note. Another student should be able to change one input and obtain the same update without guessing what the model meant.

Contents · Previous chapter · Next chapter

Use censoring and the smooth curve to practise sensitivity analysis. The observed calibration curve must account for right censoring before absolute errors are computed. A unit censored at 350 has unknown 500-hour status. Before accepting the result, identify the quantity that would change the conclusion most. Naive binary smoothing biases both quantiles. Use a survival-aware curve.

A simple baseline is essential. It reveals whether the advanced method is improving accuracy or only adding complexity. Compare model calls, wall time, uncertainty and failure-mode coverage under one declared budget. Repeat stochastic calculations with several seeds; a favourable single run is not evidence of dependable efficiency.

Students can extend the example by adding correlation, a second failure mode or a modest measurement error. Explain which part of the formula changes and which remains fixed. This exercise shows how mathematics transfers from a clean textbook case to engineering and computing without pretending that the classroom model is a certified design tool.

A second primary source is https://pubmed.ncbi.nlm.nih.gov/32548928/. Compare the problem definition, estimator and validation rather than quoting one result in isolation.

Contents · Previous chapter · Next chapter

Build a small counterexample for case-mix dependence. Quantiles depend on where the validation population’s predictions lie. The initial example is: A homogeneous fleet has few extreme-risk units. Then create a nearby case where the usual shortcut becomes unreliable. Transporting the model changes the error distribution. Describe prediction ranges.

Counterexamples are positive learning tools. They reveal the conditions that make a method work and help students recognise when a software output deserves a pause. Keep the counterexample small enough to plot or enumerate. Check sign, support, bounds and conservation of probability before adding complexity.

When writing the conclusion, separate a documented fact from an interpretation and from an illustrative scenario. Cite the primary method paper for the factual mechanism, use the local calculation for the illustration, and label any recommendation as conditional on the stated model. This keeps career and engineering advice age-appropriate and honest.

Contents · Previous chapter · Next chapter

Connect sparse high-error regions to a real decision while protecting the boundary between calculation and judgement. A high E90 may arise from genuine miscalibration or unstable smoothing in a sparse tail. Few units have predicted survival below .1. First verify the arithmetic, then translate it into an allowed action and an action the evidence does not support. The metric cannot distinguish cause alone. Show density and uncertainty.

The audit should include the original question, input provenance, failure convention, algorithm settings, convergence or stopping rule, and a sensitivity result. For an adaptive method, also preserve the sequence of choices. For an optimisation, preserve every constraint. For Bayesian updating, preserve the prior and observation model. These records turn a result into something another person can challenge and improve.

A final transfer question is useful: where else would the same mathematical idea appear? Density ratios appear in survey correction and machine learning, probabilistic constraints in planning, and conditional probability in medical testing. The context changes, but transparent assumptions and reproducible checks remain valuable.

Contents · Previous chapter · Next chapter

E50 is a median while ICI is a mean of absolute errors. A few large errors raise ICI more than E50. This is the central mathematical mechanism in e50 versus ici. Write the quantities and assumptions before using software, then reproduce the calculation or geometric argument in a small example. Neither dominates in every purpose. Report both with the curve. The useful habit is to keep the physical question, the probability model and the numerical method in separate columns so that a correct computation is not mistaken for complete evidence.

Did You Know? A result can be numerically stable and still answer the wrong question if the failure event, denominator or transformation changes. Test one nearby case and predict the direction of change before recalculating. Save inputs, intermediate values and units so a second reader can audit the path.

For practice, turn the example into a sensitivity check. Change one defensible input while holding the others fixed, compare the new result with the original and explain why it moved. Then construct a counterexample in which the shortcut fails. End with two sentences: what the mathematics supports, and what remains uncertain.

Contents · Previous chapter · Next chapter

Start with the operational question behind e90 versus emax. E90 is resistant to the single largest error, whereas Emax reports the maximum curve deviation in the evaluated range. In the illustrative case, One unstable tail point drives Emax but not E90 as strongly. Draw the relevant graph, density, boundary or timeline before calculating; the picture should use the same variables and sign convention as the equations. E90 can still hide the worst ten percent. Use the relevant tail summary.

A student should be able to rebuild the result without hidden settings. Record the sampling rule or optimisation criterion, the exact failure definition, and the evidence used for every distribution or parameter. Compare a transparent baseline with the advanced method. If they disagree, investigate the mechanism before deciding which answer is more credible.

This chapter belongs to a larger decision, so translate the number carefully. State one action the evidence permits and one action it does not. A parent or educator can ask which school ideas are doing the work: algebra organises relationships, probability represents uncertainty, calculus describes local change, and computing repeats the documented rules at scale.

Contents · Previous chapter · Next chapter

Treat calibration direction is lost as a model-checking problem rather than a vocabulary exercise. Absolute errors show magnitude but not whether survival is over- or underpredicted. Errors .06 can lie above or below identity. Recompute the example from the raw values and label every unit. Opposite operational consequences may share the value. Keep the signed curve.

The strongest check uses evidence the model did not choose for itself. Hold out later observations, an independent simulation set or a known-answer benchmark. Report errors where decisions are sensitive, especially near the failure boundary, instead of relying on one global accuracy score. A method that performs well on easy safe cases can still fail at the rare cases that motivated the analysis.

Now change one assumption and make a directional prediction before running the calculation. If the result moves the opposite way, inspect the sign convention, transformation, density ratio or constraint. This prediction-first routine develops transferable problem-solving skills because it connects symbolic work to a testable expectation.

Contents · Previous chapter · Next chapter

The mathematics of internal validation becomes useful only when its mechanism is explicit. Tuning and evaluating on the same fleet makes the curve and error quantiles optimistic. Consider the worked illustration: The smoother adapts to training noise. Preserve the original inputs, calculate step by step and compare with a second method whenever an analytic or high-budget reference exists. Quantiles do not fix overfitting. Use bootstrap correction.

There are three uncertainties to keep apart. Sampling or numerical uncertainty comes from finite computation. Parameter uncertainty comes from estimating inputs. Model-form uncertainty comes from simplified physics, distributions or dependence. Reducing the first does not automatically reduce the other two. A credible report names all three and shows which one drives the decision.

For a practical investigation, create a small reproducible notebook or spreadsheet with a frozen data sheet, formula sheet, diagnostics sheet and decision note. Another student should be able to change one input and obtain the same update without guessing what the model meant.

Contents · Previous chapter · Next chapter

Use external validation to practise sensitivity analysis. Apply the frozen model, then estimate the validation calibration curve and quantiles. A supplier model is tested in local operations. Before accepting the result, identify the quantity that would change the conclusion most. Recalibrating first masks original transport error. Report original performance first.

A simple baseline is essential. It reveals whether the advanced method is improving accuracy or only adding complexity. Compare model calls, wall time, uncertainty and failure-mode coverage under one declared budget. Repeat stochastic calculations with several seeds; a favourable single run is not evidence of dependable efficiency.

Students can extend the example by adding correlation, a second failure mode or a modest measurement error. Explain which part of the formula changes and which remains fixed. This exercise shows how mathematics transfers from a clean textbook case to engineering and computing without pretending that the classroom model is a certified design tool.

Contents · Previous chapter · Next chapter

Build a small counterexample for uncertainty. Bootstrap intervals should repeat censoring adjustment, curve smoothing and quantile calculation. The initial example is: Resample units and recompute E50 and E90. Then create a nearby case where the usual shortcut becomes unreliable. A percentile of fixed errors ignores curve uncertainty. Repeat the full pipeline.

Counterexamples are positive learning tools. They reveal the conditions that make a method work and help students recognise when a software output deserves a pause. Keep the counterexample small enough to plot or enumerate. Check sign, support, bounds and conservation of probability before adding complexity.

When writing the conclusion, separate a documented fact from an interpretation and from an illustrative scenario. Cite the primary method paper for the factual mechanism, use the local calculation for the illustration, and label any recommendation as conditional on the stated model. This keeps career and engineering advice age-appropriate and honest.

Contents · Previous chapter · Next chapter

Connect decision relevance to a real decision while protecting the boundary between calculation and judgement. A tail calibration error may matter more where decisions are sensitive, but E90 itself has no cost function. Maintenance triggers cluster near a risk threshold. First verify the arithmetic, then translate it into an allowed action and an action the evidence does not support. Good E90 does not prove net benefit. Add threshold analysis.

The audit should include the original question, input provenance, failure convention, algorithm settings, convergence or stopping rule, and a sensitivity result. For an adaptive method, also preserve the sequence of choices. For an optimisation, preserve every constraint. For Bayesian updating, preserve the prior and observation model. These records turn a result into something another person can challenge and improve.

A final transfer question is useful: where else would the same mathematical idea appear? Density ratios appear in survey correction and machine learning, probabilistic constraints in planning, and conditional probability in medical testing. The context changes, but transparent assumptions and reproducible checks remain valuable.

Contents · Previous chapter · Next chapter

Order ten errors, average positions five and six, and identify nearest-rank position nine. E50=.055 and E90=.09. This is the central mathematical mechanism in practice calculation. Write the quantities and assumptions before using software, then reproduce the calculation or geometric argument in a small example. Other quantile types may differ. Write the convention beside answers. The useful habit is to keep the physical question, the probability model and the numerical method in separate columns so that a correct computation is not mistaken for complete evidence.

Did You Know? A result can be numerically stable and still answer the wrong question if the failure event, denominator or transformation changes. Test one nearby case and predict the direction of change before recalculating. Save inputs, intermediate values and units so a second reader can audit the path.

For practice, turn the example into a sensitivity check. Change one defensible input while holding the others fixed, compare the new result with the original and explain why it moved. Then construct a counterexample in which the shortcut fails. End with two sentences: what the mathematics supports, and what remains uncertain.

Contents · Previous chapter · Next chapter

Start with the operational question behind parent and educator guidance. The metrics connect ordered data, medians, percentiles, absolute value and graphs. In the illustrative case, Ask why the average and median react differently to one large error. Draw the relevant graph, density, boundary or timeline before calculating; the picture should use the same variables and sign convention as the equations. Students may think E90 means ninety percent error. Translate it in words.

A student should be able to rebuild the result without hidden settings. Record the sampling rule or optimisation criterion, the exact failure definition, and the evidence used for every distribution or parameter. Compare a transparent baseline with the advanced method. If they disagree, investigate the mechanism before deciding which answer is more credible.

This chapter belongs to a larger decision, so translate the number carefully. State one action the evidence permits and one action it does not. A parent or educator can ask which school ideas are doing the work: algebra organises relationships, probability represents uncertainty, calculus describes local change, and computing repeats the documented rules at scale.

Contents · Previous chapter · Next chapter

Treat common misconceptions as a model-checking problem rather than a vocabulary exercise. E90 is a percentile of absolute calibration errors, not a confidence level, not accuracy for ninety percent of units and not Emax. E50 can be low despite a problematic tail. Recompute the example from the raw values and label every unit. Quantiles lose error direction. Use the curve and supporting metrics.

The strongest check uses evidence the model did not choose for itself. Hold out later observations, an independent simulation set or a known-answer benchmark. Report errors where decisions are sensitive, especially near the failure boundary, instead of relying on one global accuracy score. A method that performs well on easy safe cases can still fail at the rare cases that motivated the analysis.

Now change one assumption and make a directional prediction before running the calculation. If the result moves the opposite way, inspect the sign convention, transformation, density ratio or constraint. This prediction-first routine develops transferable problem-solving skills because it connects symbolic work to a testable expectation.

Continue with the Mathematics Learning Hub, the eduKate Mathematics Learning System, the approach to learning mathematics, the Singapore Mathematics curriculum overview, the cross-subject learning approach and the previous reliability article. Each destination has its own role: subject navigation, learning method, school-to-advanced progression, or continuity with the latest reliability mechanism.

Contents · Previous chapter · Next chapter

The mathematics of research and next reading becomes useful only when its mechanism is explicit. Austin, Harrell and van Klaveren adapted ICI, E50 and E90 for survival-model calibration. Consider the worked illustration: Read the paper beside the ordered-error worksheet. Preserve the original inputs, calculate step by step and compare with a second method whenever an analytic or high-budget reference exists. Report the smoother and quantile convention. Reproduce .055 and .09.

There are three uncertainties to keep apart. Sampling or numerical uncertainty comes from finite computation. Parameter uncertainty comes from estimating inputs. Model-form uncertainty comes from simplified physics, distributions or dependence. Reducing the first does not automatically reduce the other two. A credible report names all three and shows which one drives the decision.

For a practical investigation, create a small reproducible notebook or spreadsheet with a frozen data sheet, formula sheet, diagnostics sheet and decision note. Another student should be able to change one input and obtain the same update without guessing what the model meant.

Quick answers

What should I calculate first? Start with the estimand or failure event, then a simple baseline.

Does the advanced method guarantee accuracy? No. It needs input, numerical and model checks.

Which school mathematics matters? Algebra, graphs, probability, functions and careful units all contribute.

How can a student practise safely? Use fictional or open benchmark data and preserve every intermediate value.

What makes a career claim responsible? Describe how the mathematics is used without promising admission, salary or employment.

Contents · Previous chapter · Mathematics Learning Hub