Decision Boundaries
Decision Boundaries — A decision boundary is the surface a classifier draws to separate classes. Different models draw very different boundaries on the same data: logistic regression a straight line, kNN jagged islands, a tree axis-aligned boxes, an SVM a max-margin street.
Paint a spiral or two moons, then watch four models try to separate it at once — the linear model fails while kNN and the tree contort to fit. The tradeoffs, obvious in a single glance.
Four classifiers, one dataset. Drag a point or paint new ones on any panel — all four boundaries refit at once. On a spiral or circles the linear models (Logistic, SVM) fail while kNN and the tree contort to fit — the tradeoff in one glance.
Four classifiers, one dataset. Drag a point or paint new ones on any panel — all four boundaries refit at once. On a spiral or circles the linear models (Logistic, SVM) fail while kNN and the tree contort to fit — the tradeoff in one glance.
The idea in plain words
Every classifier draws a different kind of line. Put four on the same data and the tradeoffs are obvious at a glance: logistic regression and SVM draw straight boundaries, kNN draws jagged islands, and a decision tree draws axis-aligned boxes.
On blobs they all look fine. Switch to a spiral or concentric circles and the linear models fail completely while the flexible ones contort to fit — the whole point of having more than one model.
Now, the math
Each model defines its decision region by a different rule:
- a linear score — a straight boundary.
- neighborhood size for kNN — small k means jagged regions.
▸ Show the derivation
Linear models can only separate classes a hyperplane can split; a spiral needs a curved boundary they cannot express. kNN and trees are non-parametric — they carve arbitrarily complex regions, at the cost of overfitting on sparse or noisy data.
Now Break It
Try this: The spiral breaks the linear model; a sparse dataset makes kNN jagged.
Control: Preset shape picker (spiral / XOR)
Last updated .