Polynomial Regression
Polynomial Regression — Fit curves by adding polynomial features.
What if your data curves? Polynomial regression bends the line into a curve by feeding the model powers of the input. But bend it too far and it starts chasing noise.
- Training data
- Polynomial fit
- True function
- Extrapolation zone
The idea in plain words
A straight line can only go straight. Polynomial regression bends it into a curve by feeding the model extra features — the input squared, cubed, and so on — then fitting a linear combination of them. The degree controls how many wiggles the curve is allowed.
Raise the degree and the curve threads the training points ever more tightly, but between and especially beyond them it can rocket to absurd values. That gap between fitting the data and fitting the world is overfitting, and it motivates the bias–variance tradeoff.
Now, the math
The model is linear in the coefficients but polynomial in the input:
Stacking the powers of each x into a design matrix, the least-squares fit is:
- the degree — the highest power of x, set by the slider.
- the coefficient on the j-th power of x.
- the Vandermonde design matrix of powers of each input.
▸ Show the derivation
The columns of powers of x become nearly parallel at high degree, so X⊤X is badly conditioned — the fit is computed on a normalized input to stay numerically stable. Even so, a degree-15 polynomial through 10 points interpolates every one exactly yet oscillates wildly in between, the Runge phenomenon you can trigger with the degree slider.
Now Break It
Try this: Degree 15 wiggles through every point perfectly on train data but oscillates wildly between them.
Control: Degree slider (set to maximum)
Last updated .