The Kernel Trick

ClassificationAdvanced~8 min

The Kernel Trick — Separate non-linear data by lifting it into higher dimensions.

What if no straight line can separate your classes? The kernel trick secretly lifts your data into a higher dimension where a flat boundary works — without ever computing the coordinates.

Inner ring
Outer ring
Separating plane

Kernel

RBF γ0.60

Orbit the 3D lift: raising each point to its squared radius makes the rings separable by a flat plane — that plane projects back to a circle in 2D. Drag a point on the flat panel and watch it ride up the paraboloid as its radius grows.

Kernel

RBF γ0.60

The idea in plain words

Some data no straight line can split — two concentric rings, say. The kernel trick lifts the data into a higher dimension where a flat boundary does work. Raise every point to its squared radius and the inner ring drops low, the outer ring rises high, and a horizontal plane slices cleanly between them.

The magic is that a support vector machine never has to compute those higher coordinates — a kernel function gives the needed dot products directly. Orbit the 3D lift and watch the plane project back down to a circle in 2D.

Now, the math

A kernel computes a similarity that stands in for a dot product in feature space:

K(x, x') = e^{-\gamma \lVert x - x'\rVert^2}

$K(x,x')$: the kernel — similarity between two points.
$\gamma$: RBF width — large γ makes each point’s influence tiny and local.

▸ Show the derivation

For the polynomial kernel the implicit feature map is explicit here: (x, y) → (x, y, x²+y²). A plane in that lifted space is a conic (circle/ellipse) back in 2D. The RBF kernel corresponds to an infinite-dimensional map; too large a γ lets it memorize each point as its own island — overfitting you can trigger with the slider.

Now Break It

Try this: Huge RBF gamma makes the boundary hug each point individually — overfitting islands.

Control: Gamma slider (set to maximum)

← Back to all visualizations Continue on the Learning Path →

Last updated July 3, 2026.