Clustering with K-Means and Choosing K

Concept

K-Means is a clustering method that partitions observations into K groups. Each group is represented by a centroid: the mean position of the observations currently assigned to that group. The method repeatedly assigns rows to the nearest centroid and updates each centroid until the assignments stop changing much.

It is useful for a first clustering workflow because the mechanics are concrete. It is not a machine for discovering a uniquely correct number of business segments.

Intuition

Imagine plotting customers by annual spend and purchase frequency. With K = 3, K-Means begins with three tentative centers. Every customer is assigned to whichever center is closest. Each center then moves to the average location of its assigned customers. Repeating assignment and movement creates three compact groups when the data supports that shape.

The centroid is not necessarily an actual customer. It is an average coordinate in the selected feature space. A group with an average annual spend of 3,000 and average frequency of 12 describes a center, not a person who necessarily exists.

Why Scaling Matters

K-Means commonly uses Euclidean distance. Consider age from 18 to 80 and annual_income from 20,000 to 200,000. Raw income differences can overwhelm age differences, so clusters may mostly reflect income. Standardizing makes each selected numeric feature contribute according to its variation rather than its unit size.

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

features = ["annual_spend", "purchase_frequency", "account_age_months"]
X = customers[features]

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = KMeans(n_clusters=3, random_state=42, n_init=10)
clusters = model.fit_predict(X_scaled)

fit_predict() fits centroids to the scaled data and returns one cluster identifier for each row. random_state=42 makes the initialization reproducible for this example. n_init=10 tries several initial centroid placements and keeps the best result under K-Means' objective; different initial placements can otherwise lead to different local solutions.

Inspecting a Result

Attach the identifiers to the original table, then profile rather than immediately name the groups:

customers = customers.copy()
customers["cluster"] = clusters

profile = customers.groupby("cluster")[features].mean()
print(profile)

This answers questions such as, "Which cluster has the highest average spend?" It does not prove that the cluster is valuable, loyal, or likely to respond to a campaign. Check group sizes, medians, distributions, and representative rows before assigning a human-readable description.

Choosing K

K is usually a modeling choice. Asking for two groups and asking for six groups can both produce output. The task is to choose a level of detail that is compact, stable, interpretable, and useful for the question.

Inertia summarizes how close observations are to their assigned centroids. Lower inertia means more compact groups under the K-Means objective. However, adding more clusters nearly always lowers inertia, including the extreme case in which every row has its own cluster. Minimum inertia is therefore not a sensible rule for choosing K.

The elbow method fits several values of K, records inertia, and looks for a bend where extra clusters provide smaller improvements. The bend can be unclear, especially with messy business data. Use it as evidence, not as an oracle.

inertias = []

for k in range(2, 7):
    model = KMeans(n_clusters=k, random_state=42, n_init=10)
    model.fit(X_scaled)
    inertias.append(model.inertia_)

The code creates values for a later plot or comparison. It does not itself select K. A practical team might compare K = 3 and K = 4, examine profiles, then choose the version that supports a real intervention without inventing too many fragile subgroups.

Silhouette Score, Intuitively

A silhouette score asks whether rows are generally closer to their own cluster than to nearby alternative clusters. Higher values usually suggest clearer separation. It is helpful when comparing reasonable candidate values of K, but it has assumptions too. A strong score does not establish business value, and a lower score can occur when the real population overlaps naturally.

Limitations

K-Means works most naturally for roughly compact, similarly shaped groups. It can struggle with elongated or unevenly dense groups, extreme outliers, and data where distance is not a useful concept. It requires a chosen K, is sensitive to feature scaling, and depends on initialization. These are reasons to inspect and compare results, not reasons to discard it automatically.

Outliers and Geometry

Centroids are means, so an extreme observation can pull a centroid away from the main mass of data and create a tiny cluster around itself. Investigate unusual rows, transformations, and robust preprocessing before deciding whether an outlier is an error, a separate operational category, or evidence that another method is more appropriate.

K-Means tends to fit roughly compact, centroid-oriented groups that are reasonably separated under the chosen distance. It can cut across curved crescent-shaped groups, struggle with nested or elongated structure, and give unhelpful partitions when density or size differs sharply. A poor K-Means result does not prove the data has no structure; it may mean its geometry is mismatched. Hierarchical clustering, density-based methods such as DBSCAN, or Gaussian mixture models can represent different assumptions, but should be tried because they fit a diagnosed limitation, not as automatic upgrades.

Stability, Actionability, and Choosing K

Inertia will generally decrease as K increases, so the lowest inertia does not select the correct K. Silhouette evaluates cohesion within a cluster and separation from nearby clusters, but a higher score does not guarantee stability, interpretability, ethical suitability, or business usefulness. Cluster labels such as Cluster 0 are arbitrary identifiers; labels such as "High Value Loyalists" are analyst interpretations that need profile evidence.

After fitting, inspect centroid summaries, cluster sizes, within-cluster distributions, representative rows, and whether differences are practically meaningful. Do not rely only on a centroid because its mean can hide wide variation. Stability asks whether similar groups reappear under reasonable changes. Actionability asks whether the organization can actually take a different, justified action for those groups. A stable cluster can still be useless, and an actionable-looking cluster can still be unstable.

Best Practices

Select features that describe the similarity you care about, scale when distance makes units comparable, and use a reproducible random state. Compare a small set of plausible K values with inertia, separation, stability, and domain usefulness. Keep the original data available for profiling; a cluster label without its feature context is difficult to audit.

Interview Perspective

Question: How do you choose K in K-Means?
Answer: Compare plausible values using signals such as the elbow pattern and silhouette score, then validate that profiles are stable, interpretable, and useful for the domain.
What the interviewer is testing: whether you know that a metric supports rather than replaces judgment.
Follow-up: Why does scaling matter before K-Means?

Practice Questions

1. Why would annual income dominate age in raw Euclidean distance?
2. What happens to inertia as K increases, and why does that not select K by itself?
3. A company needs three actionable campaign groups, but the elbow is ambiguous between three and four. What would you inspect next?
4. Read the code above: which data is passed to fit_predict(), and why?
5. Why is a centroid not necessarily a real customer?

Quick Quiz

1. What does a K-Means centroid represent? Answer: The mean position of the rows assigned to a cluster.
2. Are cluster labels ordered rankings? Answer: No; they are arbitrary identifiers.
3. Does the elbow method always reveal one clear K? Answer: No; it is a heuristic and can be ambiguous.

Next Lesson

Next, explore hierarchical clustering, which records nested merges rather than creating one flat partition immediately.