Choose your reading route
eduKateSG · Why Mathematics? · Gauss–Markov Theorem: Why Least Squares Is BLUE Under Its Assumptions
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 5–9
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 10–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–34
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when least squares is called BLUE? The Gauss–Markov theorem says that under a correctly specified linear model with zero-mean, uncorrelated, equal-variance errors and full-rank design, least squares has the smallest variance among linear unbiased estimators.
Did You Know? Normal errors are not required for the core BLUE result, though normality supports familiar exact t and F inference.
Full chapter index
Chapters 1–4 · Understand the question
Chapters 5–9 · Reproduce the example
Chapters 10–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–34 · Study and verify
- A spreadsheet workflow
- A coding workflow
- Applications and careers
- A student investigation
- Common misconceptions
- Connection to recent mathematics articles
- Guidance for students and parents
- Practice questions
- Frequently asked questions
- A reporting checklist
- Proving the two-observation example
- What changes when assumptions change
- Final theorem check
- Useful next reading
BLUE means best linear unbiased estimator. Best means minimum variance inside the class of estimators linear in responses and unbiased for the target.
Penn State states that least squares estimates a linear coefficient combination with variance no larger than another linear unbiased estimator. Read Penn State.
Write y=X beta+epsilon. X has full column rank, E(epsilon|X)=0 and Var(epsilon|X)=sigma²I in the classical theorem.
| a | 1-a |
|---|---|
| 0.5 | 0.5 |
For two independent equal-variance observations, a y1+(1−a)y2 has variance sigma²[a²+(1−a)²], minimised at a=0.5 with variance sigma²/2.
| Estimator | Variance factor |
|---|---|
| Equal weights | 0.5 |
| Weights 0.8 and 0.2 | 0.68 |
A linear estimator is a fixed weighted combination of responses for the given design. Nonlinear estimators are outside the comparison.
Unbiased means repeated-sampling expectation equals the target. A biased estimator can sometimes have smaller mean squared error.
Exact dependence among design columns makes individual coefficients non-unique. Constraints or reparameterisation are required.
Omitted variables, simultaneity and selection can violate E(epsilon|X)=0 and bias least squares.
Heteroscedasticity may leave coefficients unbiased but removes the ordinary efficiency claim and can invalidate usual standard errors.
Serial or cluster dependence changes the covariance and the efficient estimator. The sampling unit must be respected.
The core variance result uses moments, not normality. Exact small-sample t and F inference commonly adds normal errors.
OLS projects y onto the column space of X. Another linear unbiased estimator adds an orthogonal component whose variance contribution is nonnegative.
With known non-spherical covariance, transformation or covariance weighting gives a generalised least-squares BLUE analogue.
Robust standard errors change uncertainty estimates while leaving OLS coefficients unchanged. They do not make OLS efficient.
Ridge introduces bias to reduce variance and may improve prediction MSE, so it lies outside the BLUE class.
List target, rank, mean condition and covariance. Mark which claims come from design knowledge, diagnostics or untestable assumptions.
An estimator can be efficient under an ideal model yet fragile to outliers. Robust methods trade some ideal efficiency for stability.
Two equal-quality sensors should receive equal weights. If one is four times noisier, covariance-aware weighting beats the simple average among linear unbiased estimates.
Say that OLS is minimum variance among linear unbiased estimators under the stated Gauss–Markov conditions, then name the most doubtful condition.
It does not mean best predictor, best nonlinear estimator, lowest MSE after bias, immunity to outliers or causal truth.
Build X, calculate X^T X inverse, beta-hat, residuals and verify X^T e=0. Use labelled columns, preserve full precision until the end and cross-check the result against a second calculation.
Store the model, target family, confidence level, degrees of freedom and every intermediate quantity. Test the implementation on the worked example before applying it to new data.
Gauss–Markov reasoning supports calibration, economics, experiments, quality control and analytics. The professional skill is matching the mathematical claim to the decision rather than reporting a number without its conditions.
Simulate equal-variance observations, compare weighted means, then introduce unequal variances. Recalculate after changing one input, draw the result and explain which part changed because of data, design or inferential scope.
BLUE does not mean smallest prediction error, best biased estimator, robustness or causal identification.
The residual standard error guide explains noise scale. The VIF guide shows design-driven variance.
Students should expand BLUE and list every assumption. Parents can ask what “best” means.
- Minimise a²+(1−a)².
- Name a violation affecting efficiency.
- Explain why ridge lies outside BLUE.
### Is normality required? No for BLUE.
### Is OLS always efficient? No.
### Does unbiased mean accurate once? No.
Report the target, data unit, model, estimate, standard error, degrees of freedom, multiplier, interval or region, assumptions, diagnostics and practical threshold. Name any exploratory choices.
Differentiate a²+(1−a)²=2a²−2a+1. The derivative 4a−2 equals zero at a=1/2, and the positive second derivative confirms a minimum. At a=1/2 the variance factor is 1/2; at a=0.8 it is 0.68. Both estimators are unbiased because their weights sum to one, but the equal-weight estimator is more efficient when the errors have equal independent variance. This miniature proof contains the logic of Gauss–Markov: alternative unbiased weights add avoidable variance when the covariance model is correct.
If the first observation has variance sigma² and the second has four sigma², equal weighting is no longer efficient. Inverse-variance weighting gives more weight to the precise observation. If the errors are correlated, the covariance term also enters the variance calculation. If the mean model is wrong, unbiasedness itself may fail. These cases do not make the theorem false; they show that its conditions define the comparison. Report which covariance model is assumed, how it was supported and whether a robust or generalised least-squares sensitivity analysis changes the conclusion.
State the estimator class before using the word best. Verify rank, conditional mean and covariance assumptions, then separate coefficient efficiency from prediction performance. If heteroscedasticity or dependence is plausible, compare the ordinary result with covariance-aware or robust alternatives. Report whether coefficients, standard errors or both change. A theorem is most useful when its conditions guide the next diagnostic and the next model, rather than ending the analysis with a slogan.
Continue through the Mathematics Learning Hub and rebuild the worked example independently before using software.
Extended practice laboratory
Practice case 1: Write model
Setup: Start from the article’s Gauss-Markov worked example for case 1. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 2: Check rank
Setup: Start from the article’s Gauss-Markov worked example for case 2. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 3: State mean condition
Setup: Start from the article’s Gauss-Markov worked example for case 3. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 4: State covariance
Setup: Start from the article’s Gauss-Markov worked example for case 4. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 5: Define linear
Setup: Start from the article’s Gauss-Markov worked example for case 5. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 6: Define unbiased
Setup: Start from the article’s Gauss-Markov worked example for case 6. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 7: Define best
Setup: Start from the article’s Gauss-Markov worked example for case 7. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 8: Write OLS
Setup: Start from the article’s Gauss-Markov worked example for case 8. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 9: Verify equations
Setup: Start from the article’s Gauss-Markov worked example for case 9. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 10: Work sensor case
Setup: Start from the article’s Gauss-Markov worked example for case 10. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 11: Differentiate variance
Setup: Start from the article’s Gauss-Markov worked example for case 11. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 12: Find optimum
Setup: Start from the article’s Gauss-Markov worked example for case 12. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 13: Add heteroscedasticity
Setup: Start from the article’s Gauss-Markov worked example for case 13. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 14: Add correlation
Setup: Start from the article’s Gauss-Markov worked example for case 14. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 15: Compare GLS
Setup: Start from the article’s Gauss-Markov worked example for case 15. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 16: Compare robust SE
Setup: Start from the article’s Gauss-Markov worked example for case 16. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 17: Compare ridge
Setup: Start from the article’s Gauss-Markov worked example for case 17. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 18: Inspect outlier
Setup: Start from the article’s Gauss-Markov worked example for case 18. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 19: Inspect residuals
Setup: Start from the article’s Gauss-Markov worked example for case 19. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 20: Check conditioning
Setup: Start from the article’s Gauss-Markov worked example for case 20. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 21: Validate prediction
Setup: Start from the article’s Gauss-Markov worked example for case 21. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 22: Separate normality
Setup: Start from the article’s Gauss-Markov worked example for case 22. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 23: Reject causal claim
Setup: Start from the article’s Gauss-Markov worked example for case 23. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 24: Audit assumptions
Setup: Start from the article’s Gauss-Markov worked example for case 24. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 25: Write conclusion
Setup: Start from the article’s Gauss-Markov worked example for case 25. Task: Change the named quantity, recompute dependent values and compare with the original.
Check: Check the defining identity, preserve units and state one diagnostic or validation step before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.