Gaussian Mixture Models (EM)

Unsupervised & Dim. ReductionAdvanced~9 min

Gaussian Mixture Models (EM) — Soft-cluster data as a blend of Gaussian blobs via EM.

A Gaussian mixture models your data as a blend of bell-shaped blobs. Instead of hard assignments, each point gets a probability of belonging to each cluster, fit by the EM algorithm alternating between guessing and updating.

Points (soft-colored)
Covariance ellipse

Iteration 0 / 25

Components (K)

Drag any mean to seed the EM initialization — the fit restarts from where you drop it. Then step through the E-step (color points by responsibility) and the M-step (refit the ellipses). A bad seed can collapse a component or land in a poor local optimum.

Iteration 0 / 25

Components (K)

The idea in plain words

A Gaussian mixture models data as a blend of bell-shaped blobs. Instead of the hard assignments of k-means, each point gets a probability of belonging to each blob — a soft assignment. Points on a boundary take a blended color.

It’s fit by Expectation–Maximization: the E-step colors points by responsibility, the M-step refits each blob’s mean and covariance. Because the blobs can stretch and rotate, GMM captures elongated, tilted clusters that k-means’ rigid circles miss.

Now, the math

The E-step computes each point’s responsibility to each component:

\gamma(z_{ik}) = \frac{\pi_k\,\mathcal{N}(x_i \mid \mu_k, \Sigma_k)}{\sum_j \pi_j\,\mathcal{N}(x_i \mid \mu_j, \Sigma_j)}

$\gamma(z_{ik})$: the responsibility of component k for point i.
$\pi_k$: the mixing weight (prior) of component k.
$\Sigma_k$: the component’s covariance — the shape of its ellipse.

▸ Show the derivation

EM alternates between the E-step (responsibilities, given current parameters) and the M-step (re-estimate means, covariances, and weights, given responsibilities). Each round can only increase the log-likelihood. If a component collapses onto a single point its covariance shrinks toward a singularity and the likelihood diverges — the failure you can trigger.

Now Break It

Try this: A component collapses onto a single point, its variance shrinking to zero — a singularity.

Control: Number of components slider (set too high)

← Back to all visualizations Continue on the Learning Path →

Last updated July 3, 2026.