Hierarchical Clustering

Unsupervised & Dim. ReductionIntermediate~7 min

Hierarchical Clustering — Build a tree of clusters by repeatedly merging the closest pair.

Hierarchical clustering doesn’t need you to pick the number of clusters up front. It merges the two closest points, then the next closest, building a tree (dendrogram) you can cut at any level.

Dendrogram — drawn one join at a time; drag to cut, hover a join to link it

Clusters at cut: 24
Most recent join
Hovered join
Cut line

Linkage controls

Data

Dataset

Points per class12

Spread1.0×

Model

Linkage

Distance between two clusters = the variance a merge would add — the closest thing to a k-means objective.

Cut into clusters24

Cut height0.15

Playback

Step 0 / 23

Speed

Score every pair
Join the closest
One fewer cluster
↻

Step 0 of 23 — no joins yet — every one of the 24 points is its own cluster

Drag the dashed line on the dendrogram to cut it — the cut and the step counter are the same control. Hover any join to ring the points it merged.

Break it

The idea in plain words

Hierarchical clustering doesn’t need you to pick the number of clusters up front. It repeatedly merges the two closest groups — starting from single points — building a tree called a dendrogram. Cut that tree at any height and you get that many clusters.

Drag the cut line and the clusters reorganize live. The linkage rule (how “distance between groups” is defined) matters enormously: single linkage can chain distant points into one straggly cluster, unlike the tighter groups k-means would find.

Now, the math

Each step merges the two clusters with the smallest linkage distance:

d(A,B) = \begin{cases}\min_{i,j} d(i,j) & \text{single}\\ \max_{i,j} d(i,j) & \text{complete}\\ \text{avg}_{i,j}\, d(i,j) & \text{average}\end{cases}

$d(A,B)$: the distance between clusters A and B under the chosen linkage.
$d(i,j)$: the distance between two individual points.

▸ Show the derivation

The dendrogram’s vertical axis is the merge height. Cutting at height h keeps every merge below h and separates the rest, so lowering the cut yields more, smaller clusters. Single linkage merges on the single closest pair, so a thin bridge of points can fuse two otherwise-distinct clusters — the chaining effect.

Trace it by hand

Four points on a line: a = 0, b = 1, c = 4, d = 5 (all distances are plain gaps on the line). We build the dendrogram bottom-up and see how the linkage choice changes the final merge height.

Step 1 — all pairwise point distances
$\begin{array}{c|cccc} & a & b & c & d\\\hline a & 0 & 1 & 4 & 5\\ b & 1 & 0 & 3 & 4\\ c & 4 & 3 & 0 & 1\\ d & 5 & 4 & 1 & 0 \end{array}$
Every point starts as its own cluster; each step merges the closest pair of clusters.
Step 2 — the two closest pairs merge first
$d(a,b) = 1 \;\Rightarrow\; \{a,b\} \text{ at } h = 1, \qquad d(c,d) = 1 \;\Rightarrow\; \{c,d\} \text{ at } h = 1$
Two merges at height 1 leave two clusters: the pair on the left and the pair on the right.
Step 3 — the linkage rule decides the final merge height
$d(\{a,b\},\{c,d\}) = \begin{cases} \min = d(b,c) = 3 & \text{single}\\ \max = d(a,d) = 5 & \text{complete}\\ \text{avg} = \tfrac{3+4+4+5}{4} = 4 & \text{average} \end{cases}$
Same clusters, three different merge heights — the dendrogram's shape depends on the linkage, not just the data.
Step 4 — cut the single-linkage tree at height 2
$h = 2: \quad \{a,b\}, \ \{c,d\} \qquad (\text{final merge at } 3 > 2)$
Cutting keeps every merge below the line: 2 clusters here, 4 clusters if you cut below 1, and 1 cluster above 3.

What just happened: The merge sequence was identical, but the last merge happened at height 3, 4, or 5 depending on the linkage — and a cut at height 2 recovered the two natural pairs without ever choosing k in advance.

Now Break It

Try this: The wrong linkage criterion (single linkage) chains distant points into one straggly cluster.

Control: Linkage method selector (set to single)

What happens: Chaining effect! Single linkage strings distant points together into one straggly cluster.

Where hierarchical clustering is used

Hierarchical clustering builds a tree of nested groups, which makes it valuable whenever the relationships between items matter as much as the final partition. Biologists use it on gene expression data to produce heatmaps with dendrograms, revealing which genes and samples behave similarly across conditions. It appears in phylogenetics to reconstruct how species relate, in document organization to build topic hierarchies, and in market research to see how products or customers nest into progressively broader groups. Because the agglomerative version repeatedly merges the closest pair of clusters, it yields a full dendrogram that you can cut at any level, letting analysts explore coarse or fine groupings from a single run without committing to a number of clusters up front.

A frequent misunderstanding is treating the dendrogram's horizontal arrangement as meaningful; only the merge heights, which encode distance, carry information, and branches can be rotated freely without changing the structure. Another pitfall is ignoring the linkage choice. Single linkage tends to chain points into straggly clusters, complete linkage favors compact ones, and Ward's method targets low within-cluster variance, so the same data can yield very different trees depending on the rule. People also underestimate the cost: standard agglomerative clustering scales roughly with the square or cube of the number of points, making it impractical for very large datasets. Finally, merges are greedy and permanent, so an early mistake cannot be undone later.

Frequently asked questions

What is a dendrogram and how do I read it?

A dendrogram is the tree diagram that hierarchical clustering produces, showing how points and clusters merge. The height at which two branches join reflects how far apart those clusters were when merged, so low joins mean similar items. You choose a number of clusters by drawing a horizontal cut across the tree and counting the branches it crosses.

What is the difference between agglomerative and divisive clustering?

Agglomerative clustering is bottom-up: every point starts alone and the closest clusters are repeatedly merged until one remains. Divisive clustering is top-down: everything starts in one cluster that is recursively split. Agglomerative is far more common in practice because divisive splitting is computationally expensive.

Which linkage method should I use?

It depends on the cluster shapes you expect. Ward's method and complete linkage produce compact, roughly equal-sized clusters and are safe defaults, while single linkage can capture elongated shapes but is prone to chaining unrelated points together. Trying a few and inspecting the dendrograms is a reasonable strategy.

How is hierarchical clustering different from K-Means?

Hierarchical clustering produces a full tree of nested groupings and does not require choosing the number of clusters before running, whereas K-Means partitions data into a fixed k in one flat layer. Hierarchical methods are deterministic but scale poorly, while K-Means is fast on large data but depends on random initialization.

Does hierarchical clustering scale to large datasets?

Not well. Standard agglomerative algorithms need to compute and update distances between clusters, giving time and memory costs that grow roughly with the square of the number of points. For large datasets people often subsample, use a faster method like K-Means, or apply specialized approximate variants.

← Back to all visualizations Continue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated August 6, 2026.