Choose your reading route
eduKateSG · Why Mathematics? · Generalized Least Squares: Correlated Errors and Covariance Weighting
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 5–9
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 10–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–35
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when regression errors have unequal variances or move together? Generalized least squares replaces ordinary Euclidean distance with covariance-weighted distance, giving more influence to precise directions and accounting for correlation when the covariance model is credible.
Did You Know? Whitening transforms correlated, unequal-variance errors into a scale where ordinary least squares geometry applies.
Full chapter index
Chapters 1–4 · Understand the question
Chapters 5–9 · Reproduce the example
Chapters 10–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–35 · Study and verify
- A spreadsheet workflow
- A coding workflow
- Applications and careers
- A student investigation
- Common misconceptions
- Connection to recent mathematics articles
- Guidance for students and parents
- Practice questions
- Frequently asked questions
- A reporting checklist
- Deriving inverse-variance weights
- Correlation changes effective information
- Feasible GLS and uncertainty
- Final covariance audit
- Useful next reading
Ordinary least squares treats the error covariance as sigma²I. GLS allows a general positive-definite covariance matrix Sigma.
Statsmodels documents GLS for a general error covariance matrix, while NIST describes weighted least squares for differing response variances. Read statsmodels and NIST.
beta-hat_GLS=(X^T Sigma^−1 X)^−1 X^T Sigma^−1 y when the matrices are well-defined.
| y | Variance |
|---|---|
| 10 | 1 |
| 14 | 4 |
Two measurements 10 and 14 have independent variances 1 and 4. Inverse-variance weights normalise to 0.8 and 0.2, giving estimate 10.8 and variance 0.8.
| Method | Estimate | Variance |
|---|---|---|
| GLS | 10.8 | 0.8 |
| OLS | 12 | 1.25 |
The equal-weight mean is 12 with variance (1+4)/4=1.25, so correct precision weighting lowers variance.
A factor L with L^T L=Sigma^−1 transforms y and X. OLS on Ly and LX equals GLS on the original scale.
Diagonal Sigma gives WLS. Each observation is weighted roughly by inverse error variance when those variances are known or well estimated.
Off-diagonal covariance represents dependence across time, space, clusters or repeated measurements.
In practice Sigma is often estimated, producing feasible GLS. Uncertainty in that estimate matters, especially in small samples.
Wrong weights can reduce efficiency or distort uncertainty. A complicated covariance model is not automatically better.
GLS corrects covariance geometry, not omitted predictors, nonlinear mean structure or causal design.
Weights must correspond to variance units. Arbitrary importance scores are not automatically statistical precision weights.
Sigma must support a valid covariance geometry. Singular or nearly singular matrices require careful modelling and numerical methods.
Inspect whitened residuals as well as original residuals. Remaining dependence or changing spread signals an inadequate covariance model.
Cluster-robust or heteroscedasticity-consistent standard errors can protect inference without claiming a fully known covariance, but they do not make OLS efficient.
Name the source of heteroscedasticity or dependence, how parameters were estimated and which sensitivity models were compared.
Repeated sensor readings may share drift. A covariance model prevents correlated repeats from being counted as independent information.
State the covariance model, transformed estimation and whether conclusions are sensitive to plausible alternatives.
GLS does not guarantee correct weights, causal identification, robustness to outliers or good prediction under distribution shift.
Students can begin with inverse-variance means, then generalise from diagonal weights to full covariance matrices.
Build Sigma and its inverse, form the weighted cross-products, solve for coefficients and verify with a whitening transformation. Keep every matrix dimension, unit and intermediate quantity visible, then verify the defining identity.
Use tested linear-algebra routines, save the design and model formula, check rank and conditioning, and test against a hand-worked example.
GLS supports time series, geospatial analysis, longitudinal studies, engineering calibration and sensor fusion.
Combine two sensors with unequal variances, then add correlated repeated readings.
GLS is not simply “trust important rows more”; weights encode a covariance model that must be justified.
The Gauss–Markov guide explains efficiency. The residual standard error guide explains noise scale.
Students should distinguish diagonal weights from full covariance. Parents can ask where the weights came from.
- Recalculate the weighted mean.
- Compare OLS variance.
- Explain whitening.
### Is WLS the same as GLS? WLS is the diagonal case.
### Must Sigma be known? Often it is estimated.
### Does GLS fix omitted variables? No.
Report the target, model, data unit, assumptions, calculation, diagnostics, validation and practical threshold. Separate descriptive evidence from causal interpretation.
For independent measurements of one mean, minimise the variance sum(a_i² sigma_i²) subject to weights summing to one. A Lagrange multiplier gives a_i proportional to 1/sigma_i². With variances 1 and 4, raw precisions are 1 and 0.25, which normalise to 0.8 and 0.2. The estimate 10.8 therefore trusts the precise sensor more, for a statistical reason rather than a subjective importance score.
Repeated observations that share drift do not contribute as much independent information as their count suggests. Positive covariance makes certain averages less precise; negative covariance can make particular contrasts more precise. GLS uses the full inverse covariance to weight directions, not just rows. Display Sigma or its structured parameters so readers can see what dependence was assumed.
When Sigma is estimated, feasible GLS treats estimated covariance parameters as inputs. Large samples can make this practical, but small-sample uncertainty and model misspecification remain. Compare plausible covariance structures, inspect whitened residuals and report sensitivity. If the coefficient is stable but standard errors move, say so; if both move, explain the weighting mechanism.
Verify that Sigma is symmetric, has appropriate units and is positive definite or otherwise handled by a justified model. Explain how its parameters were estimated and whether the same data informed both mean and covariance. Compare GLS with OLS plus robust uncertainty, inspect original and whitened residuals, and stress plausible alternatives. The smallest standard error is not automatically the most credible result; it must come from a defensible covariance description. Preserve the alternative covariance fits, numerical diagnostics and decision consequences so another analyst can reproduce why the chosen weighting model was preferred. This makes the final choice transparent, testable and easier to revise.
Continue through the Mathematics Learning Hub and reproduce the worked example before using software.
Extended practice laboratory
Practice case 1: Write Sigma
Setup: Start from the article’s GLS worked example for case 1. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 2: Check dimensions
Setup: Start from the article’s GLS worked example for case 2. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 3: Invert Sigma
Setup: Start from the article’s GLS worked example for case 3. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 4: Build cross-products
Setup: Start from the article’s GLS worked example for case 4. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 5: Solve GLS
Setup: Start from the article’s GLS worked example for case 5. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 6: Work two sensors
Setup: Start from the article’s GLS worked example for case 6. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 7: Normalise weights
Setup: Start from the article’s GLS worked example for case 7. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 8: Calculate estimate
Setup: Start from the article’s GLS worked example for case 8. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 9: Calculate variance
Setup: Start from the article’s GLS worked example for case 9. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 10: Compare OLS
Setup: Start from the article’s GLS worked example for case 10. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 11: Build whitening factor
Setup: Start from the article’s GLS worked example for case 11. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 12: Transform y
Setup: Start from the article’s GLS worked example for case 12. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 13: Transform X
Setup: Start from the article’s GLS worked example for case 13. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 14: Fit whitened OLS
Setup: Start from the article’s GLS worked example for case 14. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 15: Add correlation
Setup: Start from the article’s GLS worked example for case 15. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 16: Use diagonal WLS
Setup: Start from the article’s GLS worked example for case 16. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 17: Estimate Sigma
Setup: Start from the article’s GLS worked example for case 17. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 18: Inspect whitened residuals
Setup: Start from the article’s GLS worked example for case 18. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 19: Compare robust SE
Setup: Start from the article’s GLS worked example for case 19. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 20: Check positive definite
Setup: Start from the article’s GLS worked example for case 20. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 21: Stress weights
Setup: Start from the article’s GLS worked example for case 21. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 22: Check influence
Setup: Start from the article’s GLS worked example for case 22. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 23: Reject importance weights
Setup: Start from the article’s GLS worked example for case 23. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 24: Reject causal claim
Setup: Start from the article’s GLS worked example for case 24. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 25: Write conclusion
Setup: Start from the article’s GLS worked example for case 25. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.