Hierarchical Clustering

Concept

Hierarchical clustering builds a nested record of how observations can be grouped. The common introductory approach, agglomerative clustering, starts with every row in its own cluster and repeatedly merges the two closest clusters. The result is a hierarchy rather than one immediately fixed set of labels.

This is valuable when the question is not simply "Which four groups should we use?" A product team may first want to see whether customers form a few broad families that then split into more specific behavioral groups.

Intuition: Bottom-Up Merges

Suppose five products are represented by price, purchase frequency, and return rate. At the start there are five one-product clusters. The closest two merge. Then the method considers distances among the updated clusters and merges again. Eventually every product is connected in one hierarchy.

The merge history lets us examine multiple resolutions. A low-level cut may produce many detailed groups; a higher-level cut may produce a few broad ones. Neither choice is automatically correct. It depends on whether the chosen grouping helps a real decision.

Dendrograms

A dendrogram is a tree-like visual representation of those merges. Its leaves represent individual observations or already formed groups. Branches show merges. The vertical height of a merge roughly reflects how dissimilar the groups were according to the chosen distance and linkage rule.

Drawing a horizontal line across a dendrogram creates a possible clustering: each disconnected branch below the line becomes a group. A large jump in merge height can suggest that joining two groups would combine meaningfully different structure. It is still an interpretive clue, not a proof that one cut is objectively correct.

Linkage Is Another Choice

To merge clusters, the algorithm needs a definition of distance between clusters, not just between individual rows. This definition is called linkage.

• Single linkage considers the nearest pair of points across two clusters and can chain nearby points together.
• Complete linkage considers the farthest pair and tends to prefer tighter groups.
• Average linkage uses an average pairwise distance.
• Ward linkage merges groups to limit the increase in within-group variation and is often useful with Euclidean-style compact clusters.

Different linkage choices can give different trees for the same data. That is not a bug; it is a reminder that clustering encodes a definition of similarity. Choose and document a method that makes sense for the feature representation.

A Compact scikit-learn Workflow

As with K-Means, scale compatible numeric features before distance-based clustering:

from sklearn.cluster import AgglomerativeClustering
from sklearn.preprocessing import StandardScaler

features = ["annual_spend", "purchase_frequency", "account_age_months"]
X = customers[features]

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = AgglomerativeClustering(n_clusters=3, linkage="ward")
customers = customers.copy()
customers["cluster"] = model.fit_predict(X_scaled)

This code requests three final groups. AgglomerativeClustering can still be useful when n_clusters is chosen after exploratory reasoning, but the estimator output above is a flat label for each row. A full dendrogram requires merge information and is often created with separate visualization tooling; understand its meaning before treating it as a selection machine.

K-Means vs Hierarchical Clustering

Question	K-Means	Hierarchical clustering
Main representation	Centroids and a flat assignment	Nested sequence of merges
Need a final group count	Usually choose K up front	Can inspect hierarchy before choosing a cut
Practical scale	Often convenient for larger datasets	Can become expensive as datasets grow
Useful when	Compact groups and a clear segmentation level	Multiple levels or merge structure matter

Neither method is universally better. K-Means may be a clear operational choice when a team needs a fixed number of segments. Hierarchical clustering may be more revealing for a smaller set of products or customers where relationships between broader and narrower groups matter.

Practical Interpretation

Imagine a retailer's products. A dendrogram might first separate high-return items from lower-return items, then split the lower-return group into high- and low-frequency products. That tells a richer story than four unlabeled groups, but it must be checked against product knowledge and enough data. A merge can reflect measurement scale, an outlier, or a temporary promotion rather than a durable product family.

Best Practices

Use a small, meaningful feature set and standardize compatible numeric variables. Compare linkage choices only when you can explain why they are relevant. Inspect multiple sensible cuts, profile clusters, and check whether the hierarchy is stable enough to support its intended use. Keep a sample of original rows available for domain review.

Interview Perspective

Question: What does a dendrogram show?
Answer: It visualizes the nested merge structure of hierarchical clustering; merge heights indicate relative dissimilarity under the selected method.
What the interviewer is testing: whether you understand hierarchy and its assumptions rather than only an API call.
Follow-up: When might K-Means be more practical?

Practice Questions

1. What happens first in agglomerative clustering?
2. Why can different linkage methods produce different groups?
3. A dendrogram has a large merge-height jump. What could it suggest, and what should you avoid claiming?
4. Choose K-Means or hierarchical clustering for a small catalog where teams want broad families and subfamilies. Explain why.
5. Why should annual_spend and purchase_frequency usually be scaled before this workflow?

Quick Quiz

1. What does agglomerative clustering do? Answer: It starts with individual clusters and repeatedly merges close groups.
2. Does a dendrogram force one cluster count? Answer: No; different cuts can create different numbers of groups.
3. Is hierarchical clustering always better than K-Means? Answer: No; suitability depends on data, scale, and the question.

Next Lesson

Next, move from grouping rows to representing many features with fewer principal components using PCA.