Choose your reading route
eduKateSG · Why Mathematics? · Cook’s Distance, Regression Influence, Leverage and Residuals
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 5–9
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 10–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–24
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when one data point can change an entire regression model? Cook’s distance combines residual size and leverage to estimate how strongly deleting one observation could shift all fitted coefficients together.
It is a diagnostic invitation, not an automatic deletion rule. A large value asks us to inspect the record, data-generating process and model.
Did you know? A point can have a modest residual yet still be influential when it sits far from the centre of the predictor space.
Full chapter index
Chapters 1–4 · Understand the question
Chapters 5–9 · Reproduce the example
Chapters 10–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–24 · Study and verify
Regression fits a model using all observations. Influence asks how much that fitted model depends on one case. Cook’s distance summarises the combined movement of the coefficient vector when a case is removed.
NIST gives Cook’s distance as residual squared divided by p times MSE, multiplied by h_ii/(1−h_ii)². Here p is the number of fitted parameters and h_ii is leverage. Read the NIST regression-diagnostics reference.
Let residual e=4, p=3, MSE=4 and leverage h=0.20. The residual factor is 16/(3×4)=1.3333. The leverage factor is 0.20/0.80²=0.3125. Cook’s distance is 0.4167.
| Residual | p | MSE | Leverage |
|---|---|---|---|
| 4 | 3 | 4 | 0.2 |
A large residual says the model misses the observed response. High leverage says the predictor pattern can pull the fitted surface. Cook’s distance becomes large when both ingredients matter.
| Residual factor | Leverage factor | D |
|---|---|---|
| 1.3333 | 0.3125 | 0.4167 |
The parameter count includes the intercept when one is fitted. Changing p changes the scale of the diagnostic. Compare values only under the stated model.
Mean square error estimates typical residual variance. If the fit is noisy, a residual of four units may be ordinary; if the fit is precise, it may be exceptional.
CHAPTER 7 OF 24 · Reproduce the example
7. High leverage is not automatically influential
NIST distinguishes leverage from influence. A high-leverage point lying close to the fitted model can have a small residual and limited Cook’s distance.
CHAPTER 8 OF 24 · Reproduce the example
8. A large residual is not automatically influential
A response outlier near the centre of predictor space may have low leverage. It can still matter, but its effect on all coefficients may be smaller than its vertical error suggests.
Rules such as 4/n are screening conventions, not laws. Sample size, model complexity and domain consequences matter. Compare cases, plots and refits.
The diagnostic approximates the change produced by deleting one case. Refit the model without a flagged point to see which coefficients, predictions and conclusions actually move.
Check units, transcription, sensor status, duplicate records and coding. Correct verified errors with an audit trail. A genuine unusual observation should not be erased because it is inconvenient.
Influence can signal missing curvature, interaction, subgroup structure or changing variance. A better model may explain the point without excluding it.
Deleting one row is not enough when rows belong to one person, school or device. Cluster-level influence diagnostics may better match the sampling unit.
High-leverage points often occur at unusual predictor combinations. Predictions near them may rely heavily on sparse evidence. Mark the supported range of the model.
Robust methods can reduce sensitivity to response outliers, but they do not solve every leverage problem. Compare results and explain the changed loss function.
Plot Cook’s distance by observation, residuals against fitted values and leverage against studentized residuals. Label cases rather than hiding them.
For a fitted model, calculate residual, MSE, parameter count and leverage, then evaluate the formula row by row. Verify leverage came from the same design matrix.
Use tested regression-diagnostic functions. Return case IDs, Cook’s distance, leverage, studentized residuals and DFFITS so the investigation is reproducible.
Influence diagnostics support engineering calibration, economics, education research, quality control and machine learning model checking. Analysts use them to find fragile conclusions.
CHAPTER 20 OF 24 · Explore applications
20. Connection to recent mathematics articles
The weighted-variance guide explains squared residual contributions. The Hedges’ g guide shows why standardisation needs a clear denominator.
Students should calculate the two factors separately and write an investigation plan. Parents can ask whether a flagged point is wrong, rare or evidence that the model is incomplete.
- Recalculate with h=0.50.
- Hold h fixed and double the residual.
- Explain why deleting every large Cook value can bias a study.
### Is Cook’s distance a probability? No. It is a scaled influence measure.
### Does a large value prove an error? No. It identifies a case worth investigation.
### Can a point have high leverage and low Cook’s distance? Yes, when its residual is small.
Continue through the Mathematics Learning Hub and compare Cook’s distance, DFFITS and DFBETAs on one fitted dataset.
Extended practice laboratory
Practice case 1: Fit the original model
Setup: Use the article’s Cook-distance example and change one declared input for case 1. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 2: Record case identifier
Setup: Use the article’s Cook-distance example and change one declared input for case 2. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 3: Calculate raw residual
Setup: Use the article’s Cook-distance example and change one declared input for case 3. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 4: Square the residual
Setup: Use the article’s Cook-distance example and change one declared input for case 4. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 5: Count fitted parameters
Setup: Use the article’s Cook-distance example and change one declared input for case 5. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 6: Calculate MSE
Setup: Use the article’s Cook-distance example and change one declared input for case 6. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 7: Record leverage
Setup: Use the article’s Cook-distance example and change one declared input for case 7. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 8: Calculate residual factor
Setup: Use the article’s Cook-distance example and change one declared input for case 8. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 9: Calculate leverage factor
Setup: Use the article’s Cook-distance example and change one declared input for case 9. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 10: Multiply to Cook distance
Setup: Use the article’s Cook-distance example and change one declared input for case 10. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 11: Double the residual
Setup: Use the article’s Cook-distance example and change one declared input for case 11. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 12: Increase leverage
Setup: Use the article’s Cook-distance example and change one declared input for case 12. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 13: Compare 4 over n screen
Setup: Use the article’s Cook-distance example and change one declared input for case 13. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 14: Plot all cases
Setup: Use the article’s Cook-distance example and change one declared input for case 14. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 15: Refit without case
Setup: Use the article’s Cook-distance example and change one declared input for case 15. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 16: Compare coefficients
Setup: Use the article’s Cook-distance example and change one declared input for case 16. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 17: Compare predictions
Setup: Use the article’s Cook-distance example and change one declared input for case 17. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 18: Check transcription
Setup: Use the article’s Cook-distance example and change one declared input for case 18. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 19: Check units
Setup: Use the article’s Cook-distance example and change one declared input for case 19. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 20: Inspect model shape
Setup: Use the article’s Cook-distance example and change one declared input for case 20. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 21: Inspect clusters
Setup: Use the article’s Cook-distance example and change one declared input for case 21. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 22: Compare DFFITS
Setup: Use the article’s Cook-distance example and change one declared input for case 22. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 23: Avoid automatic deletion
Setup: Use the article’s Cook-distance example and change one declared input for case 23. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 24: Write an influence conclusion
Setup: Use the article’s Cook-distance example and change one declared input for case 24. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.