Choose your reading route
eduKateSG · Why Mathematics? · Regression Leverage, the Hat Matrix and Predictor Space
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 5–9
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 10–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–24
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when a predictor combination sits far from the rest of the data? Regression leverage measures how strongly each row’s predictor position helps determine its own fitted value. The diagonal of the hat matrix turns geometry into an auditable diagnostic.
Leverage concerns the inputs, not the response error. It helps students see where a model is interpolating and where it is being pulled toward the edge of its evidence.
Did you know? Leverages sum to the number of fitted parameters, so the average leverage is p/n.
Full chapter index
Chapters 1–4 · Understand the question
Chapters 5–9 · Reproduce the example
Chapters 10–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–24 · Study and verify
A point near the centre of predictor values usually has modest leverage. A point in a sparse or extreme predictor region can strongly shape the fitted line or surface.
For ordinary least squares, H=X(X transpose X) inverse X transpose and fitted values equal Hy. NIST identifies diagonal h_ii values as leverage. Read the NIST regression-diagnostics reference.
With an intercept and one predictor, h_i=1/n+(x_i−x-bar)² divided by the total squared distance of x values from x-bar. The first term is baseline leverage; the second measures predictor extremeness.
| x values | x mean | Squared-distance sum |
|---|---|---|
| 0, 1, 2, 3 | 1.5 | 5 |
For x values 0,1,2,3, n=4, x-bar=1.5 and the squared-distance sum is 5. At x=3, leverage is 0.25+2.25/5=0.70.
| Baseline | Distance term | h |
|---|---|---|
| 0.25 | 0.45 | 0.7 |
With intercept and slope, p=2. The four leverages are 0.70,0.30,0.30,0.70 and sum to 2. This invariant catches matrix and indexing errors.
Under the standard full-rank projection, leverage lies between 0 and 1. NIST notes a minimum related to 1/n and an average p/n.
The hat matrix projects observed responses onto the column space of X. A diagonal value measures how much the i-th response contributes to its own fitted value.
Changing y while keeping X fixed leaves leverage unchanged. This distinguishes predictor-space unusualness from residual unusualness.
A carefully designed experiment may intentionally include boundary points. They can be essential for identifying slopes. The question is whether they are valid and adequately supported.
Influence needs both leverage and disagreement with the fitted model. Cook’s distance and DFFITS combine these ingredients in different ways.
Conventions such as two or three times average leverage can flag cases, but they are not universal decision rules. Model structure and design matter.
A row can have ordinary values on each variable yet an unusual combination across variables. Multivariate leverage can detect that pattern even when pairwise plots do not.
Rare categories and interactions can create high leverage. Interpret the design matrix, not only the original labels.
Predictions far outside the observed predictor cloud may have high uncertainty even though there is no observed row to diagnose. Leverage teaches the broader lesson of staying inside supported regions.
Near-dependent predictor columns can make coefficient estimates unstable and affect geometry. Condition diagnostics and VIF complement leverage.
Changing units alone should not change the fitted subspace when the transformation is handled consistently. Numerical conditioning can still improve after centring and scaling.
For simple regression, use the scalar formula. For multiple regression, rely on matrix-capable software and check that leverage sums to p.
Use a tested linear-model routine, preserve row IDs and return leverage with residual and influence diagnostics. Verify full rank and intercept handling.
Leverage analysis supports experimental design, sensor calibration, economics, ecology and AI regression pipelines. It shows where evidence is sparse but consequential.
CHAPTER 20 OF 24 · Explore applications
20. Connection to recent mathematics articles
The Cook’s-distance guide combines leverage with residual size. The pooled-standard-deviation guide develops degrees-of-freedom reasoning.
Students should describe the predictor position before interpreting a leverage value. Parents can ask whether edge cases are errors, planned boundary measurements or unsupported extrapolations.
- Verify all four leverages in the worked example.
- Add x=10 and predict which leverage grows.
- Explain why changing y does not change H.
### Is leverage the same as influence? No. Leverage describes predictor position; influence describes model change.
### Can leverage be negative? Not for ordinary diagonal hat values.
### Does high leverage mean delete? No. Validate and investigate.
Use the Mathematics Learning Hub and plot predictor space, leverage and Cook’s distance together.
Extended practice laboratory
Practice case 1: Build design matrix
Setup: Use the article’s regression-leverage example and change one declared input for case 1. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 2: Include intercept
Setup: Use the article’s regression-leverage example and change one declared input for case 2. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 3: Calculate X transpose X
Setup: Use the article’s regression-leverage example and change one declared input for case 3. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 4: Invert cross-product
Setup: Use the article’s regression-leverage example and change one declared input for case 4. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 5: Form hat matrix
Setup: Use the article’s regression-leverage example and change one declared input for case 5. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 6: Extract diagonal
Setup: Use the article’s regression-leverage example and change one declared input for case 6. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 7: Sum leverage values
Setup: Use the article’s regression-leverage example and change one declared input for case 7. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 8: Check p over n average
Setup: Use the article’s regression-leverage example and change one declared input for case 8. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 9: Use simple-regression shortcut
Setup: Use the article’s regression-leverage example and change one declared input for case 9. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 10: Calculate predictor mean
Setup: Use the article’s regression-leverage example and change one declared input for case 10. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 11: Calculate squared-distance sum
Setup: Use the article’s regression-leverage example and change one declared input for case 11. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 12: Move one x outward
Setup: Use the article’s regression-leverage example and change one declared input for case 12. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 13: Change y only
Setup: Use the article’s regression-leverage example and change one declared input for case 13. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 14: Add a rare category
Setup: Use the article’s regression-leverage example and change one declared input for case 14. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 15: Add an interaction
Setup: Use the article’s regression-leverage example and change one declared input for case 15. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 16: Inspect predictor cloud
Setup: Use the article’s regression-leverage example and change one declared input for case 16. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 17: Compare influence
Setup: Use the article’s regression-leverage example and change one declared input for case 17. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 18: Check extrapolation
Setup: Use the article’s regression-leverage example and change one declared input for case 18. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 19: Check matrix rank
Setup: Use the article’s regression-leverage example and change one declared input for case 19. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 20: Centre predictors
Setup: Use the article’s regression-leverage example and change one declared input for case 20. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 21: Scale units
Setup: Use the article’s regression-leverage example and change one declared input for case 21. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 22: Compare partial leverage
Setup: Use the article’s regression-leverage example and change one declared input for case 22. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 23: Avoid automatic deletion
Setup: Use the article’s regression-leverage example and change one declared input for case 23. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.
Practice case 24: Write a leverage conclusion
Setup: Use the article’s regression-leverage example and change one declared input for case 24. Task: Recalculate from raw values, preserve units and state which mathematical or design condition changed.
Check: Check the relevant invariant, separate the number from its claim, and name the next diagnostic or source of evidence. Rebuild the relevant counts before interpreting the coefficient, preserve the stated design, and write one sentence separating the numerical result from the claim it can support. Then vary one defensible assumption, calculate again, and explain whether the conclusion changes because of new data, a new reference model, or a new decision purpose. Finish by naming the unit, tolerance, and evidence needed for transfer. Record one graph or table that would expose failure hidden by the summary.