A model is judged not only by what it predicts, but by what it cannot explain.
The difference between an observed outcome and the model’s prediction is called a residual. It sounds like a technical leftover. In practice, residuals are one of the most useful places to look when we want to know whether a model is quietly lying to us.
A good model does not need every residual to be zero. Real systems contain noise, omitted detail and irreducible variation. What matters is whether the residuals behave like the model says they should behave. Random-looking leftovers can be acceptable. Structured leftovers are often evidence that something important remains outside the model.
This article is a specialist branch beneath How Models Work. The root explains purpose, simplification, assumptions, testing and correction. Residual analysis is one of the principal ways we discover whether that simplification preserved the relationships that mattered.
Observed Minus Predicted
In a simple regression, a residual can be written as:
residual = observed value − predicted value
A positive residual means the observation sat above the model’s prediction. A negative residual means it sat below.
For one observation, that difference tells us little. Across hundreds or thousands of cases, the shape of the residual field can become a diagnostic map.
Random Scatter Is Often What We Want
If a model is well specified for its intended job, residuals plotted against fitted values or predictors should often look broadly patternless around zero.
Patternless does not mean aesthetically perfect. It means there is no obvious curve, funnel, block, trend or sequence that the model failed to capture.
The moment a pattern appears, the residuals are telling us that the model’s “unexplained” part contains structure.
A Curve Means the Relationship May Be Wrong
Suppose a straight-line model is fitted to a relationship that is actually curved.
The residuals may be positive at low values, negative in the middle and positive again at high values. The model looks reasonable overall while the residual plot quietly sketches the curve the model refused to include.
This is a classic form of misspecification: the chosen functional form is too simple for the structure in the data.
A Funnel Means the Variance Is Changing
If residual spread increases as fitted values rise, the model may be assuming constant variance where the real process becomes noisier at larger values.
This matters because uncertainty estimates, standard errors and prediction intervals can all become misleading when the error structure is wrong.
The model may still predict the centre reasonably well while being overconfident in one part of the range and underconfident in another.
Residuals Can Reveal Time Dependence
If residuals alternate, drift or cluster across time, errors may not be independent.
Yesterday’s miss may contain information about today’s miss. A forecasting model that repeatedly underpredicts after one type of event and overpredicts after another has left temporal structure outside itself.
The same issue appears in space. Nearby locations can share residual structure because geography creates dependence the model did not represent.
See How Geography Works | Spatial Autocorrelation for the geographic owner.
An Outlier Is a Question, Not a Verdict
A large residual may indicate data entry error, measurement failure, an unusual but valid case, a missing variable, a regime change or a genuinely rare event.
The weakest response is to delete the point because it makes the chart look untidy.
A large residual should trigger investigation: why was this case so hard for the model? Sometimes the case is wrong. Sometimes the model is wrong. Sometimes the world simply produced something the model was never designed to cover.
Residuals Are Model-Relative
The same observation can have a large residual under one model and a small residual under another.
This means residuals are not errors floating independently in nature. They are discrepancies between the world and a particular model representation.
If the model changes, the residual field changes too. That makes residuals powerful for comparing model adequacy, but it also means we should never speak of “the residual” without naming the model that produced it.
Worked Example: Housing Prices
Suppose a model predicts housing prices using floor area alone.
The residuals may show that homes near transport nodes are consistently more expensive than predicted, while homes far from amenities are consistently cheaper. That pattern tells us location has been omitted from the model.
Add location and the residual pattern may shrink. New patterns can still remain: age, floor level, lease condition or market regime may matter next.
Residual analysis is therefore not one final inspection. It is a conversation between model and world.
Worked Example: MRT Journey Times
A timetable model predicts journey time from distance and scheduled dwell.
If residuals grow systematically during peak periods, the model may be missing crowding effects, dwell variability or network interaction. If residuals cluster on one segment, the missing mechanism may be local infrastructure or operating constraints.
The railway-specific owner remains How MRT Works | It’s Mathematics. Residuals add the model-diagnostic layer: where does observed reality repeatedly disagree with the simplified timetable model?
A Careful Analogy: Learning
A teacher has an implicit model of a learner: “this student understands fractions.” Then the learner repeatedly fails only when variables are introduced.
Those repeated mismatches are educational residuals in an analogical sense. The teacher’s model of the learner is too coarse. The errors are structured rather than random.
The repair is not to blame the residual. It is to revise the model of what the learner actually knows.
Residuals Can Be Small and Still Misleading
A flexible model can drive residuals very low on the data it was fitted to and still generalise badly.
That is why training residuals alone do not prove model quality. Cross-validation, held-out testing and external validation ask whether the apparent fit survives new data.
See How Cross-Validation Works and How Research Validity Works.
A Residual Diagnostic
- Define the prediction and observation on the same scale.
- Plot residuals against fitted values.
- Plot residuals against important predictors.
- Check time and spatial ordering where relevant.
- Look for curvature, funnels, clusters and regime changes.
- Investigate large residuals rather than deleting them reflexively.
- Compare residual structure across plausible models.
- Validate on data the model did not train on.
- Revise the model only when the added complexity improves the intended job.
Read the Mechanism in Three Directions
Forward: data → model prediction → residual → pattern check → model revision. Backward: start from a recurring model failure and ask what residual pattern should have warned us earlier. Across: compare statistician, domain expert and receiver; the same residual can mean noise to one person and a costly systematic failure to another.
Residuals are the part of reality the model has not successfully turned into explanation. Their pattern tells us whether that remainder is ordinary noise or missing structure asking to be modelled.
Continue through How Models Work, How Regression Works and the master How X Works hub. Next: model misspecification — what happens when the form of the model itself cannot represent the mechanism we need.