Understanding K-Means Clustering
K-means turns a table of numeric measurements into K groups. Each row represents an observation, and each column represents a feature used to measure similarity. Imagine positions on a warehouse floor: nearby positions may form useful groups for planning storage zones. K-means needs no known group labels, but you still choose the features, their relative importance, and the number of groups. Its results describe a distance-based partition, not a discovery of objectively correct categories.
Each group is represented by a centroid: the coordinate-wise mean of its assigned observations. A centroid need not be an actual observation. With starting centroids selected, the algorithm alternates two operations. First, assign every observation to its nearest centroid using Euclidean distance. Second, replace each centroid with the mean of its assigned observations. Repeat until the convergence criterion is met or the iteration limit is reached. This alternating procedure is commonly called Lloyd’s algorithm.
The objective is to minimize inertia: the sum, over all observations, of squared distances to their assigned centroids. Squaring makes distant observations contribute disproportionately, so outliers can pull centroids away from dense groups. Each assignment or mean-update step cannot increase this objective in exact arithmetic, but the procedure can settle into a local solution. Different starting centroids may therefore produce different partitions. Multiple initializations improve the search; they do not guarantee the globally best partition.
Distance also explains the importance of scaling. If one feature spans thousands and another spans single digits, the first may dominate assignments. Standardization can balance their numerical scales, but equalizing variance is a modeling choice, not proof that both features deserve equal importance. K-means works best for compact, roughly spherical groups with reasonably comparable spread. Curved groups, strongly unequal densities, and irrelevant features can make its partitions misleading.
In scikit-learn, cluster identifiers are arbitrary integers, not rankings or class meanings. The fitted centers define a rule for assigning new observations, but every observation receives an assignment, even if it lies far from all centers. Thus, K-means is not inherently an anomaly detector. Interpret centers and cluster sizes alongside domain knowledge rather than treating a small inertia as sufficient evidence of usefulness.
Practical walkthrough
Use Python 3.10 or later with NumPy and scikit-learn 1.2 or later already installed in a compatible environment. The example creates all data in memory and uses no external resources. Familiarity with Python lists and two-dimensional arrays is enough: an array of shape (4, 2) contains four observations measured on two features.
Start with four warehouse positions measured in meters along two perpendicular axes. Two positions lie near the origin; two lie farther away. We choose K = 2 to demonstrate the mechanics, not because four observations establish a reliable operating policy. Both columns use the same unit, and preserving physical distance is intentional, so this example does not standardize them.
Supply two explicit initial centers, one in each visible group. This makes the arithmetic and cluster numbering reproducible without relying on random initialization. Set n_init=1 because there is only one supplied starting configuration. In ordinary exploratory work, use k-means++ initialization with several restarts, such as n_init=10, and a fixed random_state for repeatability. Restarts cost additional computation but reduce dependence on an unfortunate start.
Fit using only X_train. Inspect labels_, cluster_centers_, and inertia_, then call predict on separate new positions. Prediction uses the existing centers; it does not update them. These new positions demonstrate assignment, not test accuracy.
For a larger study, compare plausible K values using training or validation data, examining inertia, silhouette scores, and interpretability together. Keep final test observations out of model selection. If scaling is needed, fit the scaler on training observations only and reuse it unchanged for validation and prediction.
Worked example
import numpy as np
from sklearn.cluster import KMeans
# Coordinates in meters; physical distance is the intended metric.
X_train = np.array([
[0.0, 0.0],
[0.0, 2.0],
[10.0, 10.0],
[10.0, 12.0],
])
initial_centers = np.array([[0.0, 0.0], [10.0, 10.0]])
model = KMeans(
n_clusters=2,
init=initial_centers,
n_init=1,
algorithm="lloyd",
random_state=42,
)
model.fit(X_train)
X_new = np.array([[1.0, 1.0], [9.0, 11.0]])
print("Training labels:", model.labels_.tolist())
print("Centers:", model.cluster_centers_.round(1).tolist())
print(f"Training inertia: {model.inertia_:.1f}")
print("New labels:", model.predict(X_new).tolist())
Expected output
Training labels: [0, 0, 1, 1]
Centers: [[0.0, 1.0], [10.0, 11.0]]
Training inertia: 4.0
New labels: [0, 1]
The first assignment places the first two rows together and the last two together. Averaging each pair moves the centers to (0, 1) and (10, 11). Reassigning points leaves these groups unchanged. Every training position is exactly one meter from its center, so inertia is four squared meters: four contributions of 1². Each new position is also one meter from its nearest center, producing identifiers 0 and 1 without changing training inertia. The explicit initialization fixes center ordering here; random_state has no effect on these supplied starts. Swapping center order would swap identifiers without changing the geometric partition.
Common mistakes and how to avoid them
Choosing the largest K because it has the lowest inertia.
More centers allow lower optimal training inertia. Look for diminishing improvements, inspect silhouette scores and cluster sizes, and prefer a useful partition over automatically maximizing K. An elbow may be ambiguous or absent.
Standardizing the complete dataset before reserving test observations.
Split first, then fit preprocessing on training data only. A pipeline containing StandardScaler and KMeans keeps prediction transformations consistent when the pipeline itself is fitted only on training observations.
Treating cluster 1 as a higher category than cluster 0.
Inspect feature summaries and centers before naming groups. Integers identify centers only; compare partitions independently of identifier numbering when assessing stability across fits.
Assuming a fixed random seed guarantees a good solution.
A seed supports reproducibility, not quality. Use multiple starts, inspect sensitivity across fits, and reconsider the features or clustering method if the resulting groups remain unstable.
Review questions
Why does the centroid update use a mean rather than a median?
For fixed assignments, the coordinate-wise mean minimizes the sum of squared Euclidean distances. A median corresponds to a different distance objective, so substituting it changes the algorithm.
Does predict learn from the new positions?
No. It assigns each position to the nearest fitted center. Learning new centers requires another fitting procedure; prediction alone leaves the fitted model unchanged.
Why might converting one coordinate from meters to centimeters change clusters?
That coordinate becomes numerically 100 times larger, multiplying its squared-distance contribution by 10,000. Restore consistent units or deliberately choose feature scaling before fitting.
What does low inertia establish, and what does it not establish?
It establishes compactness under the chosen features, scaling, and K. It does not establish meaningful categories, causal relationships, or good handling of unseen observations far from every center.
