Skip to content
ML Visualization

Linear Regression

RegressionBeginner~7 min

Linear Regression — Linear regression is a supervised learning algorithm that fits the best straight line through data to predict a continuous value, choosing the slope and intercept that minimize the squared error (ordinary least squares).

Linear regression finds the single straight line that gets as close as possible to all your data points at once. It’s the simplest model — and often the first one you should try.

  • Data points
  • Fitted line
  • Residuals
0.50
3.00
Mean squared error1.484

The idea in plain words

Linear regression draws the one straight line that sits closest to all the points at once. “Closest” has a precise meaning: measure each point’s vertical distance to the line (its residual), square those distances so big misses count more, and pick the line that makes the total as small as possible.

Because every point pulls on that total, a single far-out leverage point can drag the whole line toward it — the same outlier sensitivity you can feel in loss functions. When there’s no closed form, the line is found with gradient descent.

Now, the math

The model is a straight line:

y^=β1x+β0\hat{y} = \beta_1 x + \beta_0

It minimizes the mean squared error, which has a closed-form slope:

β1=∑i(xi−xˉ)(yi−yˉ)∑i(xi−xˉ)2\beta_1 = \frac{\sum_i (x_i - \bar{x})(y_i - \bar{y})}{\sum_i (x_i - \bar{x})^2}
y^\hat{y}
the predicted value (a point on the line).
β1\beta_1
the slope — how much ŷ changes per unit of x.
β0\beta_0
the intercept — the prediction when x = 0.
xˉ, yˉ\bar{x},\ \bar{y}
the mean of the inputs and outputs.
▸ Show the derivation

Setting the derivative of MSE with respect to each β to zero yields the “normal equations,” whose solution is the closed form above. Because it depends on sums over every point, one extreme leverage point can shift the slope substantially — the fragility you can trigger by adding an outlier.

Now Break It

Try this: Add one extreme leverage outlier — it drags the entire regression line off course.

Control: Click to add a point far from the cluster

← Back to all visualizationsContinue on the Learning Path →

Last updated .