Gaussian Mixture Models (EM)

Unsupervised & Dim. ReductionAdvanced~9 min

Gaussian Mixture Models (EM) — Soft-cluster data as a blend of Gaussian blobs via EM.

A Gaussian mixture models your data as a blend of bell-shaped blobs. Instead of hard assignments, each point gets a probability of belonging to each cluster, fit by the EM algorithm alternating between guessing and updating.

Component 1
Component 2
Component 3
Covariance ellipse (2σ)
Ellipses about to move (E-step)

Negative log-likelihood vs iteration

GMM controls

Data

Dataset

Points per class22

Spread1.0×

Model

Components (K)

Covariance shape

Each component may take any tilted ellipse — the unconstrained fit.

EM iterations14

Drag any mean on the plot to seed its component — the fit restarts from where you drop it.

Playback

Step 0 / 27

Speed

E: re-split the memberships
M: refit the ellipses
↻

Step 0 of 27 — E-step of iteration 1: every point re-splits its membership across the 3 components — the ellipses have not moved yet. log L -190.5

Log-likelihood-190.5

Break it

The idea in plain words

A Gaussian mixture models data as a blend of bell-shaped blobs. Instead of the hard assignments of k-means, each point gets a probability of belonging to each blob — a soft assignment. Points on a boundary take a blended color.

It’s fit by Expectation–Maximization: the E-step colors points by responsibility, the M-step refits each blob’s mean and covariance. Because the blobs can stretch and rotate, GMM captures elongated, tilted clusters that k-means’ rigid circles miss.

Now, the math

The E-step computes each point’s responsibility to each component:

\gamma(z_{ik}) = \frac{\pi_k\,\mathcal{N}(x_i \mid \mu_k, \Sigma_k)}{\sum_j \pi_j\,\mathcal{N}(x_i \mid \mu_j, \Sigma_j)}

$\gamma(z_{ik})$: the responsibility of component k for point i.
$\pi_k$: the mixing weight (prior) of component k.
$\Sigma_k$: the component’s covariance — the shape of its ellipse.

▸ Show the derivation

EM alternates between the E-step (responsibilities, given current parameters) and the M-step (re-estimate means, covariances, and weights, given responsibilities). Each round can only increase the log-likelihood. If a component collapses onto a single point its covariance shrinks toward a singularity and the likelihood diverges — the failure you can trigger.

Trace it by hand

One E-step by hand: the point x = (1, 0) under two Gaussians with means mu1 = (0,0) and mu2 = (4,0), equal isotropic covariances Sigma = 1.5 I, and equal mixing weights pi = 0.5 (values rounded to 5 decimal places, responsibilities to 3).

Step 1 — density under component 1 (the near one)
$\mathcal{N}(x \mid \mu_1, \Sigma_1) = \frac{\exp\left(-\tfrac{1}{2}\cdot\tfrac{1}{1.5}\right)}{2\pi \cdot 1.5} = \frac{\exp(-0.333)}{9.425} = 0.07603$
With Sigma = 1.5 I the exponent is just the squared distance over 2 times 1.5, and the normalizer is 2 pi times 1.5.
Step 2 — density under component 2 (the far one)
$\mathcal{N}(x \mid \mu_2, \Sigma_2) = \frac{\exp\left(-\tfrac{1}{2}\cdot\tfrac{9}{1.5}\right)}{2\pi \cdot 1.5} = \frac{\exp(-3)}{9.425} = 0.00528$
Being 3 units away instead of 1 shrinks the density by a factor of e to the minus 8 thirds.
Step 3 — weight each density by its mixing weight
$\pi_1 \mathcal{N}(x \mid \mu_1, \Sigma_1) = 0.5 \times 0.07603 = 0.03801, \qquad \pi_2 \mathcal{N}(x \mid \mu_2, \Sigma_2) = 0.5 \times 0.00528 = 0.00264$
With equal weights the comparison is really between the two densities; unequal weights would tilt it.
Step 4 — normalize into responsibilities
$\gamma(z_{i1}) = \frac{0.03801}{0.03801 + 0.00264} = 0.935, \qquad \gamma(z_{i2}) = 1 - 0.935 = 0.065$
This is the soft assignment: x belongs 93.5 percent to blob 1 and 6.5 percent to blob 2 — no hard cutoff.

What just happened: The point 1 unit from one mean and 3 units from the other got responsibilities of 0.935 and 0.065 — a graded membership k-means cannot express. The M-step would next reuse exactly these numbers as weights when refitting each blob.

Now Break It

Try this: A component collapses onto a single point, its variance shrinking to zero — a singularity.

Control: Number of components slider (set too high)

What happens: Singularity! A Gaussian collapsed onto one point — its variance went to zero and likelihood exploded.

Where gaussian mixture models (em) is used

Gaussian Mixture Models describe data as a blend of several Gaussian components, giving each point a soft, probabilistic membership across clusters rather than a single hard label. This soft assignment is valuable when groups genuinely overlap, such as separating speakers in audio, modeling the distribution of pixel intensities for image segmentation, or estimating subpopulations in financial returns. Because a GMM is a full probability model fitted by the Expectation-Maximization algorithm, it can serve as a density estimator: you can score how likely a new observation is under the learned mixture, which supports anomaly detection when a point has very low probability. Its ability to fit elliptical, differently oriented and sized components makes it more flexible than centroid clustering when the real groups are not neatly spherical.

A key misconception is that GMMs are just a fancier K-Means; in fact K-Means is a special case that emerges when covariances are forced to be equal spheres and assignments are made hard. GMMs instead learn each component's shape through a covariance matrix, which is why they capture stretched, tilted clusters. The main pitfalls come from EM: it converges only to a local optimum and depends on initialization, so multiple restarts help. With too many components or unconstrained covariances the likelihood can diverge as a Gaussian collapses onto a single point, which regularization or shared covariance structures prevent. You still choose the number of components yourself, often guided by criteria like BIC or AIC rather than by the algorithm.

Frequently asked questions

What does the EM algorithm actually do?

Expectation-Maximization fits the mixture in two alternating steps. In the E-step it computes, for each point, the probability of belonging to each Gaussian component given the current parameters; in the M-step it updates each component's mean, covariance, and weight using those probabilities. Repeating these steps increases the data likelihood until it converges.

How is a GMM different from K-Means?

K-Means assigns each point to exactly one cluster and assumes spherical, equal-sized groups, while a GMM gives soft probabilistic memberships and learns each cluster's elliptical shape through a covariance matrix. K-Means is effectively a restricted GMM with hard assignments and identical spherical covariances. GMMs are more flexible but slower and more sensitive to initialization.

What is soft clustering?

Soft clustering means each point receives a probability of belonging to every cluster rather than a single definitive label. A point sitting between two Gaussian components might be sixty percent one and forty percent the other. This is useful when clusters overlap and a hard boundary would be misleading.

How many Gaussian components should I use?

The number of components is a choice you make, not something EM discovers. Information criteria such as BIC and AIC balance fit against complexity and are commonly used to compare models with different component counts. Domain knowledge about how many subpopulations plausibly exist should also guide the decision.

Why does my GMM sometimes fail or give errors?

A frequent cause is a component collapsing onto a few points, driving its covariance toward zero and the likelihood toward infinity. This numerical instability is worsened by too many components or too little data per component. Adding a small regularization term to the covariance, reducing components, or constraining covariance structure usually fixes it.

← Back to all visualizations Continue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated August 6, 2026.