Did you know? One of the simplest ways an AI system can make a useful prediction is to learn a straight line. When a measured quantity tends to increase or decrease as another quantity changes, linear regression provides a starting model that is easy to calculate, explain and test.
Quick answer: gather appropriately measured examples → choose the target and features → fit numerical weights and an intercept → calculate predictions → measure error on new cases → check whether a straight-line relationship is appropriate. The mathematics can be simple; trustworthy use still depends on the data and the problem.
Imagine a fictional library trying to estimate how many books might be borrowed as the number of visitors changes. More visitors may often mean more borrowed books. A simple linear model can express that relationship using a line, but it cannot guarantee the outcome on a particular day.
Linear regression in 30 seconds
- Feature: an input measurement, such as the number of visitors.
- Target: the numerical quantity to predict, such as books borrowed.
- Weight or coefficient: the numerical change associated with one unit of a feature in the model.
- Intercept or bias: the model’s predicted value when its numerical inputs are zero.
- Residual: the difference between an observed target and a predicted value.
- Loss function: a rule for measuring how badly predictions fit the training examples.
- Generalisation: useful performance on genuinely new observations.
Linear regression predicts a number. Logistic regression, despite its related name, is commonly used for binary classification probabilities. They solve different types of questions.
A fictional library example
Suppose a simplified toy library model is predicted books = 2 × visitors + 5. This is not a fitted real-world relationship; its numbers are invented solely to demonstrate the formula.
If there are ten visitors, the prediction is 2 × 10 + 5 = 25 books. If there are fifteen visitors, the prediction becomes 2 × 15 + 5 = 35 books.
The weight 2 means the model adds two predicted books for each additional visitor, while other information remains fixed. The intercept 5 is what the formula predicts at zero visitors.
That zero-visitor result might be physically unrealistic in a library. It is a warning that a mathematical intercept does not always have a sensible interpretation in the real world, especially when zero lies outside the training range.
Step 1: Decide exactly which quantity matters
A model’s target needs a clear definition. “Will the library have a good day?” is vague. “How many books will be borrowed between noon and closing?” is much more precise.
The feature should also be measured consistently. A visitor count from an automated entry sensor may not mean the same thing as the number of unique borrowers.
A system can optimise a perfectly defined loss against the wrong target. Before fitting, ask whether the model’s quantity supports the decision people actually need to make.
Step 2: Collect trustworthy pairs of numbers
Training requires example inputs paired with observed outcomes. For our fictional library, one row might contain 10 visitors, 25 books; another might contain 15 visitors, 35 books.
But do not treat the toy formula as historical evidence. Real borrowing depends on opening hours, events, collection availability, staff operations and other conditions. A useful data collection must consider those influences.
Measurement mistakes matter too. A scanner outage or a duplicate visitor count could distort the fitted relationship. Linear regression cannot know which records were recorded incorrectly without suitable checks.
Step 3: Plot the data before fitting the line
A scatterplot shows each example as a point, with the feature on one axis and the target on another. The points might form an upward trend, downward trend or no simple pattern at all.
A line is most intuitive when the relationship is approximately linear over the range that matters. If the points form a strong curve, or consist of several distinct groups, a single straight line may be inadequate.
The Google Machine Learning Crash Course: Linear Regression introduces how a fitted line uses learned weights and a bias to represent numerical relationships.
Step 4: Understand the prediction formula
A one-feature linear regression can be written as prediction = weight × feature + bias, often shown as ŷ = wx + b.
For our toy model, w = 2, b = 5, and x is the visitor count. The predicted output changes linearly as x changes.
The same idea extends to multiple features: prediction = b + w₁x₁ + w₂x₂ + …. A library might include the visitor count, scheduled event indicator and opening hours, provided the features are available at prediction time and encoded appropriately.
A coefficient measures how the model associates a feature with its target conditional on the other specified model features. It does not automatically tell us what would happen if we deliberately changed that feature in the real world.
Step 5: Learn the slope from examples
In the simplest two-point toy problem, suppose one observation is (10,25) and another is (15,35). The slope of the line through them is (35 − 25) ÷ (15 − 10) = 10 ÷ 5 = 2.
With a slope of 2, we can calculate the intercept from the first observation: 25 − 2 × 10 = 5. That gives our illustrative equation ŷ = 2x + 5.
Real regression usually considers many observations rather than choosing a line through just two points. Measurements are rarely perfectly aligned, and the fitted coefficients depend on the objective and data.
Step 6: Why a prediction can miss the observed value
Imagine the model predicts 25 books for ten visitors but the library actually records 22. The residual, defined here as actual minus predicted, is 22 − 25 = −3 books.
A residual of −3 indicates that this prediction exceeded the observation by three. Another day might have a positive residual if the actual total exceeds the prediction.
Looking only at whether residuals average to zero can hide problems: overpredictions and underpredictions may cancel. That is why developers use measures based on absolute errors or squared errors.
Step 7: Measure error with mean squared error
Mean squared error (MSE) squares each prediction error, then takes the average. Squaring makes both positive and negative mistakes contribute positively, and it penalises larger errors more strongly.
Take three fictional prediction errors of 2, −3 and 1 books. Their squares are 4, 9 and 1. The MSE is (4 + 9 + 1) ÷ 3 = 14/3 ≈ 4.67 squared books.
The unit is squared because the calculation squares the errors. The square root of MSE—called RMSE—returns to the original unit and is easier to interpret in some contexts.
Google’s linear-regression loss module explains why different loss functions respond differently to errors.
Step 8: What ordinary least squares does
A common linear regression procedure chooses coefficients that minimise the sum of squared residuals on training data. This is called ordinary least squares.
For suitable problems, the optimal coefficients can be calculated using linear algebra rather than repeatedly exploring random lines. In other workflows, an optimiser such as gradient descent adjusts the weights and bias iteratively.
The scikit-learn LinearRegression reference describes the ordinary least-squares estimator, while the Google gradient-descent guide explains iterative parameter updates.
There is no contradiction: different methods can solve a related optimisation problem. The best numerical method depends on the data and implementation.
Step 9: How gradient descent changes the line
Suppose our starting line underpredicts larger visitor counts. Gradient descent calculates how the chosen loss changes as its weight and intercept change.
The optimiser then takes a numerical step in a direction expected to lower the loss, using a chosen learning rate. The process repeats until suitable stopping or convergence criteria are met.
A learning rate that is too large can make optimisation unstable; one that is too small may take excessive time. For a beginner-friendly walkthrough, read How Gradient Descent Teaches AI From Errors.
Step 10: Test the line on observations it has not seen
A line can fit its training points without predicting future library activity well. A meaningful evaluation separates training examples from independent validation or test examples.
For a time-ordered forecasting task, the information boundary matters: a model making a Monday prediction cannot use Tuesday’s recorded outcomes as input. Tests may need chronological rather than arbitrary splits.
Compare the fitted line against a simple baseline, such as a historical average that was available at the time. If the added complexity produces no consistent improvement, the simpler approach may be preferable.
Step 11: Inspect the residual pattern
Plot residuals against predicted values or input features. A repeating curved pattern may indicate that one straight line is not capturing the relationship.
If errors grow with visitor count, the model’s reliability may differ between quiet and busy days. If one special event creates an extreme observation, investigate whether it is a legitimate unusual condition or a recording error.
Residual analysis cannot settle every problem, but it can reveal weaknesses that a single average error metric conceals.
The important question is not “Did we fit a line?” It is “Does this line remain useful for the decisions and observations that matter?”
Step 12: Beware of extrapolation
Interpolation predicts within a range represented by relevant training data. Extrapolation predicts beyond it.
If the fictional training records involve ten to forty visitors, predicting for a thousand visitors is a very different problem. The linear formula continues mechanically, but the real library could run out of books, space or staff capacity.
An AI model should not quietly turn a line into an unlimited law of nature. Explain when a prediction falls far outside the data that supported the model.
Step 13: What if there are outliers?
One unusually high borrowing day can strongly influence a least-squares fit because squared error magnifies large deviations.
Some alternatives use different loss functions or robust regression methods. But an unusual observation is not automatically a mistake: it might reflect a genuine school event that the model should recognise.
Review the original data and intended use before discarding examples. A robust method does not replace thoughtful measurement and domain knowledge.
Step 14: Do correlated features create problems?
Suppose the model includes both “number of visitors” and “number of people entering the library” measured in nearly the same way. Those variables may be strongly correlated.
When features carry overlapping information, individual coefficients can be unstable or difficult to interpret, even if the model’s predictions remain useful in some settings. This is related to multicollinearity.
A developer may simplify the features or consider methods such as ridge regression under appropriate conditions. Avoid assigning a causal story to one coefficient just because it has a positive sign.
Step 15: Linear regression versus logistic regression
Despite the shared word regression, these methods commonly target different outputs.
Linear regression estimates a numeric quantity, such as the number of books. Logistic regression commonly models a binary event probability—for example, whether a defined threshold will be exceeded—using a different output transformation and loss.
Confusing the two can cause misleading interpretations of a model’s outputs. The correct choice begins with what the target actually represents.
Step 16: When can linear regression be useful?
- Demand estimation: explore relationships between properly measured quantities.
- Educational demonstrations: learn slopes, intercepts, errors and numerical predictions.
- Scientific baselines: compare more elaborate models against a transparent starting point.
- Business analytics: examine associations with appropriate controls and cautious interpretation.
- Model auditing: see whether extra complexity improves outcomes on a genuine test.
These are possible uses, not permission to invent observations. A model should be reported with its data scope and limits.
Try it: fit a line on paper
Create three fictional points: (1,3), (2,5), (3,7). Draw them on graph paper and notice that they follow the line y = 2x + 1.
Predict the value for x = 4: 2 × 4 + 1 = 9. Now change the last observed point to (3,8) and ask whether one line still passes through all three points.
You have begun to see why regression works with an objective rather than expecting every real observation to lie perfectly on the same line.
Five mistakes to avoid
- “A coefficient proves causation.” Predictive association and real-world intervention are different.
- “A fitted line is valid forever.” Relationships and operating conditions can change.
- “Zero input gives a meaningful real-world intercept.” Zero may lie outside the observed range.
- “Training accuracy is enough.” Separate test results determine generalisation.
- “A straight line must fit every pattern.” Curves, groups and changing relationships require investigation.
Frequently asked questions
What does linear regression predict?
A numerical quantity using a learned weighted combination of input features and an intercept.
Is linear regression considered AI?
It is a standard statistical and machine-learning method, and machine learning is part of the broader AI field.
What is a residual?
The difference between an observed target and the model’s predicted value, under a stated sign convention.
Can linear regression learn from multiple features?
Yes. Multiple linear regression combines several numerical or appropriately encoded features with learned coefficients.
Does a low training error guarantee a good forecast?
No. The model needs independent evaluation, realistic inputs and checks for changes in conditions.
Your one-minute takeaway
Linear regression is a transparent starting point for AI prediction: it learns a weighted line or linear relationship, checks error and produces numerical estimates. Its simplicity makes it valuable for teaching and for baselines. Its limits—especially causation, extrapolation, data leakage and changing conditions—must remain visible.
Trusted reading and connected articles
- Google Machine Learning Crash Course: Linear Regression — weights, bias and numerical predictions.
- Google: Linear Regression Loss — squared error and model evaluation.
- Google: Gradient Descent — learning the coefficients.
- scikit-learn: LinearRegression — the ordinary least-squares estimator.
- How Gradient Descent Teaches AI From Errors — understand the updates.
- How AI Forecasts Sales, Traffic and Demand — more forecasting context.
- How AI Is Tested Before Release — independent evaluation.
- Secondary 1 Mathematics Tutor Clementi — mathematical understanding from first principles.
- How AI Works: Complete Overview — continue exploring.