Did you know? A computer can group similar examples without being told their category names. But sometimes a single group label is not enough. Four objects might naturally form two pairs, and those two pairs might also belong to one larger family. Hierarchical clustering is a way to organise these nested relationships.
Quick answer: represent each item with consistent features → choose a distance and linkage rule → begin with individual items or broad groups → repeatedly merge or split groups → draw a dendrogram → choose a useful cut → test whether the hierarchy is stable and meaningful.
This is usually an unsupervised-learning method: the algorithm does not need pre-existing class labels. It finds a structure under the chosen distance and merge rules. It does not thereby discover someone’s true identity or an objectively correct category.
Hierarchical clustering in 30 seconds
- Observation: one item to be grouped.
- Feature vector: numerical representation of that item.
- Distance metric: a rule that compares observations.
- Cluster: a set of observations grouped together.
- Linkage: a rule determining the distance between entire clusters.
- Agglomerative: bottom-up clustering that merges smaller groups.
- Divisive: top-down clustering that splits larger groups.
- Dendrogram: a tree diagram showing the sequence and height of merges.
Start with four numbers, not a complicated AI dashboard
Imagine four fictional data points on a number line: A = 1, B = 2, C = 8 and D = 9. To keep the example visible, let distance mean absolute numerical difference.
A and B are one unit apart. C and D are also one unit apart. The two pairs are much farther from each other.
A bottom-up algorithm begins with four one-item clusters: {A}, {B}, {C}, {D}. It can first join A and B, then join C and D. Because the two nearest pairs tie at distance 1, either of these pair merges can be shown first.
Step 1: Set the feature meaning
An observation may represent a sensor reading, document vector or product measurement. For a school demonstration, four numbers are sufficient. For real work, features must describe comparable quantities.
If one feature is measured in thousands and another between 0 and 1, a raw Euclidean distance can be dominated by the larger-scale feature. Make a deliberate decision about scaling and make it using training or analysis data appropriately.
If a variable represents a category rather than a magnitude, assigning it arbitrary numbers such as 1, 2 and 3 and calculating ordinary distance may falsely imply an order.
Step 2: Decide what counts as ‘near’
With the example A = 1 and B = 2, their absolute distance is |1 − 2| = 1. With B = 2 and C = 8, the distance is 6. Those are facts about the numeric representation, not about the intrinsic worth or meaning of the items.
Different projects may use Euclidean distance, Manhattan distance, cosine-related dissimilarity or other measures under suitable assumptions. The same dataset can yield different clusters under different geometry.
Step 3: Merge the nearest individual clusters
Agglomerative clustering starts with each observation in its own cluster. With our four-point example, one possible tie-breaking order is to combine A and B at height 1, then C and D at height 1.
Now there are two clusters: {A, B} and {C, D}. To decide how or when they merge, the algorithm needs a linkage rule. This is where the mechanism becomes especially important.
Step 4: Understand single, complete and average linkage
- Single linkage: cluster distance is the smallest distance between one point from each cluster.
- Complete linkage: cluster distance is the largest such pairwise distance.
- Average linkage: cluster distance is the average of the pairwise distances across the two clusters.
- Ward linkage: uses a variance-based merging objective; it should not be described as just another generic pairwise-distance summary.
For {A, B} = {1, 2} and {C, D} = {8, 9}, the four cross-cluster distances are 7, 8, 6 and 7. Single linkage uses the minimum, 6. Complete linkage uses the maximum, 8. Average linkage uses (7 + 8 + 6 + 7) / 4 = 7.
The same raw points can therefore produce different final merge heights. Changing linkage changes the meaning of cluster-to-cluster similarity; the choice is not a cosmetic graph setting.
Step 5: Draw the dendrogram
A dendrogram looks like a branching tree. Each original observation appears at a leaf. Branches join at heights determined by the clustering method.
For average linkage in this four-point example, A joins B at height 1, C joins D at height 1, and the two resulting pairs join at height 7. Higher branches indicate merges at greater dissimilarity under the selected rule.
A dendrogram is not automatically a biological family tree or an evolutionary history. Its branches represent the grouping rule and available measurements, not ancestral descent.
Step 6: Cut the tree to choose a working number of groups
A horizontal cut between heights 1 and 7 in our example produces two groups: {A, B} and {C, D}. A cut above height 7 produces one group. A cut below height 1 leaves all four observations separate.
The dendrogram lets a reader explore more than one grouping resolution. However, the largest visible jump in height does not always reveal a uniquely correct number of real-world categories.
Step 7: Know why single linkage can create chains
Imagine a string of points, each fairly close to its immediate neighbour but stretching across a long distance. Single linkage can join the entire chain because it only checks the closest bridge between clusters.
This may be valuable for some irregular shapes but unhelpful when the goal is to find compact, clearly separated groups. Complete or average linkage may behave differently. Use the linkage rule that matches the intended interpretation.
Step 8: Ward linkage is a different design choice
Ward’s method selects merges that minimise the increase in within-cluster squared variation. It often favours relatively compact, balanced clusters under Euclidean geometry.
The scikit-learn hierarchical clustering documentation distinguishes Ward, single, complete and average linkage and explains their different behaviours.
Do not choose Ward while casually assuming every arbitrary distance metric is supported in the same way. Geometry and linkage assumptions belong together.
Step 9: Recognise the effect of outliers
A very distant observation may remain isolated until a high merge level. That pattern can prompt investigation, but it does not prove the item is wrong, fraudulent or unimportant.
Check measurement errors, rare-but-valid events, missing data and the scale used. A one-off point can be an artefact of preprocessing rather than evidence of a special group.
Step 10: Assess whether the grouping persists
Clusters discovered in one noisy sample can change if a few observations or features are altered. Re-run the analysis with sensible perturbations, compare plausible feature sets and examine whether important relationships remain reasonably stable.
Quantitative measures such as silhouette scores may help in suitable cases, but no one score proves that a cluster has a natural-language meaning. Domain evidence must still be supplied.
Step 11: Know the difference from k-means and classification
Our existing guide to clustering without labels introduces unsupervised grouping. K-means typically seeks a fixed number of clusters around centroids. Hierarchical clustering offers a nested sequence of possible groupings instead.
A support vector machine is different again: it learns a boundary from labelled training examples. A hierarchy made without labels is not automatically a supervised classifier.
Some agglomerative clustering implementations build a hierarchy for the supplied dataset and do not include a direct prediction rule for a completely new item. Applying the result in production may require a separately specified assignment policy.
Step 12: Consider the computing cost
Storing or calculating pairwise distances can become expensive as the number of observations grows. Some algorithms have demanding memory and time requirements, particularly without suitable neighbourhood or connectivity restrictions.
For extremely large datasets, specialised approximate, streaming or non-hierarchical methods may be more practical. A pretty tree is not proof the computation scales.
Where could hierarchical clustering be useful?
- Exploratory science: comparing measured samples or feature profiles when no trusted categories are supplied.
- Document exploration: finding nested relationships among carefully constructed text representations.
- Educational data demonstrations: showing how distance rules create different groupings without assigning permanent labels to children.
- Quality control: examining whether numerical measurement profiles have compact families worth further investigation.
Applications should include privacy and interpretation safeguards. A cluster is not a diagnosis, a ranking of people, or evidence of cause.
Common mistakes and repairs
- ‘A tree diagram proves a natural family.’ The hierarchy depends on feature choice, metric and linkage.
- ‘Linkage types are interchangeable.’ Single, complete, average and Ward optimise different merge criteria.
- ‘The number of clusters is already fixed.’ A dendrogram can be cut at several resolutions.
- ‘A large branch height is an objective social distinction.’ It is a distance under a selected numerical representation.
- ‘Hierarchical clustering automatically predicts new labels.’ Prediction for new samples depends on the specific tool and assignment method.
Try it with a pencil
- Write four points on a number line: A = 1, B = 2, C = 8, D = 9.
- Circle A with B and C with D. Both pair distances equal 1.
- Calculate cross-pair distances 7, 8, 6 and 7.
- Find the average-linkage distance: 7. Draw a final branch joining the two pairs at height 7.
- Draw a horizontal cut at height 4. How many clusters appear? Two.
- Now ask what changes if you select single linkage. The final merge height becomes 6, even though the four original numbers remain unchanged.
Frequently asked questions
What is hierarchical clustering in machine learning?
It is a method for building nested groups of observations using a specified similarity, distance and merging or splitting rule.
What is a dendrogram?
A tree-like chart of the hierarchical grouping process, with heights representing merging dissimilarities under the chosen method.
Does hierarchical clustering need labelled data?
Typically no. Agglomerative hierarchical clustering is a standard unsupervised technique.
Is Ward linkage always the best choice?
No. Ward’s variance-based criterion suits some Euclidean clustering tasks, while other data shapes or distance measures may call for different linkage rules.
Is a cluster the same thing as a factual category?
No. A cluster is a mathematical pattern found under assumptions. It requires independent interpretation.
The one-minute takeaway
Hierarchical clustering starts with items or groups and constructs a nested tree of relationships. Distance and linkage rules determine the merges; a dendrogram shows how those merges unfold. The strength of the result depends on sensible features, scale, interpretability, stability and honest limits—not on the visual appeal of the tree.
Trusted reading and connected articles
- scikit-learn: Clustering and Hierarchical Methods — agglomerative clustering, dendrograms and linkage choices
- scikit-learn: Preprocessing — feature scaling and transformed distances
- How AI Finds Patterns Without Labels — the wider unsupervised-learning overview
- How AI Reduces Data Dimensions With PCA — a related but distinct feature-space tool
- How Support Vector Machines Separate Categories — supervised boundary learning
- How AI Handles Personal Data and Privacy — responsible analysis
- How AI Works: Complete Overview — connected educational articles
- Secondary 1 Mathematics Tutor Clementi — understanding numerical relationships and error checking