Recap: Hierarchical Clustering

DS701 Session 5 — Mon Sep 21, 2026

Today’s plan

  • 5 min — knowledge-check review
  • 20 min — highlights and Q&A (answer or pass — answering always earns credit)
  • 60 min — in-class activity (small groups)
  • wrap-up and cold-call check-ins

Knowledge-Check Review

KC 1: The procedure and the picture

Describe the agglomerative hierarchical clustering procedure in three steps, and say what the height at which two branches join in a dendrogram represents.

KC 2: min versus max

Single linkage defines the distance between two clusters as the distance between their closest pair of points; complete linkage uses their farthest pair. Explain, from these definitions alone, why single linkage tends to produce long chains that can connect two well-separated groups through a few noise points, and why complete linkage tends toward compact clusters of similar diameter.

KC 3: When the client will not tell you \(k\)

A client asks you to segment their customers but cannot tell you how many segments they want. Explain why hierarchical clustering is a natural fit here compared to \(k\)-means, and how you would still get a concrete number of segments out of it.

Highlights

Not one partition — a whole tree

Last session: a strict partition — every point in exactly one of \(k\) clusters, \(k\) chosen in advance.

  • But “how many clusters?” often has several honest answers, depending on scale.
  • A hierarchical clustering returns nested clusters organised as a tree.
  • The dendrogram records the containment relations: leaves are points, each join is a merge, height is the distance at which it happened.
  • It does not decide the number of clusters — you cut it, at any level, afterwards.

The agglomerative procedure

  1. Initialise: each point is its own cluster; compute the pairwise distance matrix if not given.
  2. Merge the two closest clusters (per the linkage and the metric).
  3. Update the distances from the new cluster to all remaining clusters.
  4. Repeat 2–3 until one cluster remains.

The output is a linkage matrix of shape \((n-1, 4)\) — one row per merge:

\[ [\ \text{idx}_1,\ \text{idx}_2,\ \text{distance},\ \text{count}\ ] \]

Indices \(0 \dots n-1\) are original points; indices \(n, n+1, \dots\) are the clusters created by earlier rows. scipy.cluster.hierarchy.linkage produces it; dendrogram draws it; fcluster cuts it.

Everything hinges on the linkage

Distance between two clusters \(C_i, C_j\) — three natural choices, plus one borrowed from \(k\)-means:

Single \[\min_{x\in C_i,\,y\in C_j} d(x,y)\]

Complete \[\max_{x\in C_i,\,y\in C_j} d(x,y)\]

Average \[\frac{1}{|C_i||C_j|}\sum_{x\in C_i,\,y\in C_j} d(x,y)\]

Ward: the increase in within-cluster sum of squares if \(C_i\) and \(C_j\) were merged — merge the pair that hurts the \(k\)-means objective least. A hierarchical, greedy \(k\)-means.

Same data, four personalities

linkage good at fails when shape bias
single connected, odd shapes (moons, rings); unequal sizes a few noise points bridge two groups → chaining none
complete balanced, similar-diameter clusters; ignores bridges one far outlier dominates the max compact / spherical
average a compromise; less noise- and outlier-sensitive — elliptical
ward like \(k\)-means: robust, tidy blobs non-convex shapes; needs Euclidean features elliptical

No linkage wins every row — the linkage is a modelling choice about what a cluster should look like.

Cutting the tree

Two ways to flatten a hierarchy:

  • by height — fcluster(Z, h, criterion="distance"): every merge below \(h\) is done. The red and green lines are two such cuts.
  • by count — fcluster(Z, k, criterion="maxclust"): at most \(k\) clusters.
  • Where to cut? Look for a large gap between successive merge heights, or sweep \(k\) and score each cut (silhouette), or ask what the clusters are for.

Heights are only comparable within one dendrogram — single-linkage heights are nearest-neighbour distances, Ward’s accumulate squared error.

Hierarchical vs. partitional

\(k\)-means (partitional) agglomerative (hierarchical)
needs \(k\) up front yes no — cut afterwards
output one partition a tree of nested partitions
input points in \(\mathbb{R}^d\) (needs means) feature matrix or any distance / similarity matrix
cluster shape spherical Voronoi cells depends on linkage
cost cheap, iterative \(O(n^2)\) memory, \(O(n^2 \log n)\) time — fine for thousands, not millions
repeatability random init deterministic

Reach for a dendrogram when you do not know \(k\), when the nesting itself is the answer (taxonomies), or when all you have is a distance matrix. Today’s activity: the same data, both tools, and three kinds of mess — find out for yourself which survives what.

In-Class Activity

Activity: dendrograms on real data, then make it messy

Goal: build and read dendrograms for four linkages on a real dataset, cut them into flat clusters, then inject a unit change, two outliers and a chaining bridge and watch which linkages survive — and whether \(k\)-means would have been fine all along.

  • Work in groups of 2–3. Open your section’s notebook.
  • Parts marked (autograded) are submitted to Gradescope; the rest is participation.
  • Staff will circulate — be ready to explain any part of your work.
  • Dependencies: numpy, pandas, matplotlib, scipy, scikit-learn.

Section A1

Section B1

The activity notebook goes live on the day of the lecture. Colab is optional — you can also open it on GitHub and run it locally.

Going deeper

Full lecture notes: Hierarchical Clustering