Support Vector Machine
Support Vector Machine — Find the boundary with the widest margin between classes.
A support vector machine doesn’t just find any boundary — it finds the one with the widest possible margin, the biggest gap between the two classes. Only the closest points (the support vectors) matter.
- Class 0
- Class 1
- Boundary
- Margins
Drag a non-support point — nothing moves. Drag a ringed support vector and the whole street re-solves. Small C widens the margin until it swallows violations.
Drag a non-support point — nothing moves. Drag a ringed support vector and the whole street re-solves. Small C widens the margin until it swallows violations.
The idea in plain words
An SVM doesn’t just find a separating line — it finds the one with the widest possible margin, the biggest empty street between the two classes. Only the closest points, the support vectors, touch that street and determine it.
Drag a point far from the boundary and nothing changes. Drag a support vector and the whole street re-solves. When the classes overlap, the soft-margin parameter C decides how many violations to tolerate for a wider margin — the bridge to the kernel trick.
Now, the math
The SVM maximizes the margin, equivalently minimizing ‖w‖ subject to the labels:
- the weight norm — smaller means a wider margin.
- soft-margin strength — large C punishes violations, small C tolerates them.
▸ Show the derivation
The support vectors are the points with margin ≤ 1; the solution depends only on them, which is why moving other points does nothing. Trained here with Pegasos — sub-gradient descent on the hinge loss plus an L2 term whose weight is set by C.
Now Break It
Try this: Tiny C ignores misclassifications and picks a huge sloppy margin; huge C overfits to every point.
Control: C (regularization) slider
Last updated .