K-Means Clustering

Unsupervised & Dim. ReductionBeginner~7 min

K-Means Clustering — K-means is an unsupervised clustering algorithm that partitions data into k groups by alternating between assigning each point to its nearest centroid and moving each centroid to the mean of its members, minimizing within-cluster variance.

K-means finds groups in your data by repeating two simple steps: assign each point to the nearest center, then move each center to the middle of its group. Repeat until nothing changes.

Clustering…

Cluster 1
Cluster 2
Cluster 3
Centroid

Inertia vs iteration

Clustering controls

Data

Dataset

Points per class22

Spread1.0×

Model

k (clusters)3

Starting centroids

Or drag any centroid on the plot to place it yourself.

Playback

Step 0 / 0

Speed

Assign
Update
↻

Step 0 of 0

Inertia (within-cluster sum of squares)—

Break it

The idea in plain words

K-means finds groups by repeating two steps until nothing changes: assign each point to its nearest center, then move each center to the average of its points. Step through the iterations and watch the centers slide into place.

Because the objective isn’t convex, a bad starting placement can converge to an obviously wrong grouping — crowd the centers in one corner to see it. Unlike k-nearest neighbors, there are no labels; k-means discovers structure on its own.

Now, the math

It minimizes the within-cluster sum of squares (inertia):

J = \sum_{i} \lVert x_i - \mu_{c(i)} \rVert^2

$x_i$: a data point.
$\mu_{c(i)}$: the centroid of the cluster point i is assigned to.
$J$: total inertia — smaller means tighter clusters.

▸ Show the derivation

The assign and update steps each never increase J, so the algorithm always converges — but only to a local minimum. In practice you run it several times from different starts (or use k-means++) and keep the lowest-inertia result.

Trace it by hand

Five points — (1,1), (2,1), (1,2), (6,5), (7,6) — and k = 2 centroids starting at mu1 = (0,0) and mu2 = (4,4). One assign step and one update step, with squared Euclidean distances throughout (fractions kept exact; decimals rounded to 2 places).

Step 1 — assign each point to its nearest centroid
$\begin{array}{c|cc|c} x_i & \lVert x_i-\mu_1\rVert^2 & \lVert x_i-\mu_2\rVert^2 & c(i)\\\hline (1,1) & 2 & 18 & 1\\ (2,1) & 5 & 13 & 1\\ (1,2) & 5 & 13 & 1\\ (6,5) & 61 & 5 & 2\\ (7,6) & 85 & 13 & 2 \end{array}$
Each point simply picks the smaller squared distance: three points join cluster 1, two join cluster 2.
Step 2 — total the winning distances: inertia
$J = 2 + 5 + 5 + 5 + 13 = 30$
J sums each point's squared distance to its own centroid. The starting placement scores 30.
Step 3 — move each centroid to the mean of its points
$\mu_1 = \left(\tfrac{1+2+1}{3},\ \tfrac{1+1+2}{3}\right) = \left(\tfrac{4}{3},\ \tfrac{4}{3}\right), \qquad \mu_2 = \left(\tfrac{6+7}{2},\ \tfrac{5+6}{2}\right) = (6.5,\ 5.5)$
The update step is just an average per cluster — no distances involved.
Step 4 — re-assign and watch the inertia fall
$J = \tfrac{2}{9} + \tfrac{5}{9} + \tfrac{5}{9} + \tfrac{1}{2} + \tfrac{1}{2} = \tfrac{7}{3} \approx 2.33$
No point changes cluster, so the algorithm has converged — inertia dropped from 30 to 2.33 in a single round.

What just happened: One assign-update round cut the inertia from 30 to 2.33 and nothing moved afterwards: each step provably never increases J, which is why k-means always settles — though only into a local minimum.

Now Break It

Try this: Bad initial centroid placement gets stuck in a terrible local minimum — clusters are obviously wrong.

Control: Drag centroids to adversarial starting positions (e.g., all in one corner)

What happens: Stuck in a local minimum! The algorithm converged, but the clusters are clearly wrong. Initialization matters.

Where k-means clustering is used

K-Means clustering is a workhorse for turning raw records into actionable groups. Retailers use it for customer segmentation, splitting shoppers by spending, frequency, and recency so marketing can target each group differently. It compresses images by reducing millions of pixel colors to a small palette of representative centroids, and it powers document grouping when text is first turned into numeric vectors. Engineers use it as a fast preprocessing step to summarize sensor readings or to initialize more complex models. Because it scales to large datasets and runs quickly, K-Means is often the first clustering method people reach for when they need a rough but useful partition of unlabeled data into a chosen number of groups.

A common misconception is that K-Means discovers the correct number of clusters on its own. It does not; you must supply k in advance, and different values produce entirely different partitions, so techniques like the elbow method or silhouette scores are needed to choose it sensibly. Another pitfall is assuming the algorithm handles any cluster shape. K-Means minimizes squared distance to centroids, which biases it toward roughly spherical, similarly sized groups, so it struggles with elongated or nested shapes. Results also depend on initialization and feature scaling: unscaled features let large-magnitude columns dominate, and a single run can settle into a poor local optimum, which is why running k-means++ initialization several times is standard practice.

Frequently asked questions

How do I choose the number of clusters k?

There is no single correct answer, so you evaluate several candidate values. The elbow method plots within-cluster variance against k and looks for the point where added clusters stop helping much, while silhouette scores measure how well-separated the clusters are. Domain knowledge often matters more than any metric, since the useful number of segments depends on what decision the clustering will inform.

Why do I get different results each time I run K-Means?

K-Means starts from randomly chosen initial centroids and can converge to different local optima depending on where it begins. Running the algorithm multiple times and keeping the lowest-error solution reduces this variability. Smart initialization such as k-means++ spreads starting centroids apart, which makes good outcomes far more likely.

Do I need to scale my features before clustering?

Usually yes. K-Means relies on distances, so a feature measured in thousands will overwhelm one measured in fractions, distorting the clusters. Standardizing features to comparable ranges, for example zero mean and unit variance, lets every dimension contribute fairly.

Can K-Means handle categorical data?

Not directly, because averaging category labels to form a centroid is not meaningful. You can one-hot encode categories, but distances then behave oddly, so a variant called k-modes or k-prototypes is often better for categorical or mixed data. For purely categorical problems, consider methods designed for that data type.

What is the difference between K-Means and KNN?

They are unrelated despite the similar names. K-Means is unsupervised clustering that groups unlabeled data into k clusters, while K-Nearest Neighbors is a supervised method that classifies or predicts a new point using the labels of its closest known examples. One discovers structure; the other uses existing labels to make predictions.

← Back to all visualizations Continue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated August 6, 2026.