Imagine one thousand points on a page.
At first they look like noise.
Then somebody draws circles around them.
Three groups appear.
The drawing may reveal structure.
It may also create more certainty than the points deserve.
Quick Read
Clustering groups observations according to similarity, density, connectivity or another structural rule. Common families include centroid-based methods such as k-means, hierarchical methods that build nested groupings, and density-based methods such as DBSCAN that can discover irregularly shaped groups and label some points as noise.
One-sentence answer: clustering is lossy because a rich set of individual observations is compressed into a smaller vocabulary of group labels, centroids or branches, preserving broad structure while discarding some individual variation and uncertainty.
A Cluster Is Not Something the Data Always “Has”
Some datasets contain obvious separated groups.
Others form gradients, rings, chains or overlapping clouds.
The clustering algorithm brings assumptions about what counts as a group.
K-means prefers compact groups around centroids. DBSCAN looks for density-connected regions. Hierarchical clustering depends on a distance metric and linkage rule.
Change the algorithm and the ontology can change.
k-Means: Reduce Many Points to Centroids
Choose a number of clusters, k.
Assign points to nearby centroids.
Move the centroids toward their assigned points.
Repeat.
The result compresses each cluster into a representative centre plus membership assignments.
That is useful because the centre summarises. It is lossy because no centroid can preserve every member’s location, shape, outlier status or internal substructure.
Choosing k Is Already a Model Decision
Tell k-means to find three groups and it will find three.
Tell it to find six and it will find six.
Metrics such as inertia, silhouette score and gap-style reasoning can help, but there may be no single natural k.
A clean cluster count can be an operational choice rather than a discovered law of nature.
Distance Metrics Decide What “Similar” Means
Euclidean distance treats straight-line geometry as meaningful.
Cosine similarity focuses on direction more than magnitude.
Manhattan distance accumulates coordinate differences differently.
For categorical or mixed data, other distance definitions may be required.
Clustering does not begin with groups. It begins with a definition of similarity.
Scaling Can Change the Clusters
If one feature ranges from 0 to 1 and another from 0 to 1,000, distance calculations can become dominated by the second feature unless scaling is handled deliberately.
Normalisation therefore sits upstream of many clustering pipelines.
The grouping can change because the representation changed before clustering even began.
Hierarchical Clustering Keeps More Than One Possible Resolution
A dendrogram does something beautiful.
It does not force one final grouping immediately. It shows nested merges or splits across levels.
Cut the tree high and you get a few broad groups. Cut it lower and those groups divide.
This preserves more multiscale structure than a single flat label, but the linkage rule still shapes the tree.
DBSCAN Lets Some Points Remain Noise
Density-based clustering can form groups from dense regions and leave isolated points unassigned.
This is useful when forcing every point into a cluster would be misleading.
But density parameters matter. Change the neighbourhood radius or minimum points and the number and shape of clusters can change substantially.
Outliers Are Often the First Things a Cluster Label Hides
A point at the edge of a cluster receives the same label as one sitting at the centre.
Yet those two observations may have very different certainty of membership.
Hard labels compress proximity into category.
Soft clustering or posterior membership probabilities can preserve more uncertainty when the application needs it.
Clusters Can Be Stable or Fragile
Resample the data.
Change the initialisation.
Remove one feature.
Adjust the scaling.
If the clusters reorganise completely, that instability is part of the result.
A polished plot should not hide fragile structure.
Projection Can Manufacture Apparent Clusters
A two-dimensional embedding may visually separate groups that overlap in the original high-dimensional space.
This is why clustering a projection and clustering the original feature space are not automatically equivalent.
The projection has already performed a lossy transformation.
Clusters discovered afterward inherit that transformation.
Cluster Labels Can Become Stereotypes
“High-value customer.”
“Weak learner.”
“Urban user.”
“At-risk patient.”
A label compresses a group’s statistical tendency into a portable identity.
Once the label travels farther than the original data, people begin treating group averages as personal truths.
The representation becomes socially stronger as it becomes informationally smaller.
Education: Clustering Can Help Teachers—If the Groups Stay Temporary
A teacher may notice three recurring error profiles.
One group struggles with vocabulary. Another with question interpretation. Another with execution under time pressure.
Grouping allows targeted intervention.
But students move. A child can belong to one profile in comprehension and another in writing. A cluster should guide the next teaching move, not become the student’s permanent identity.
Science Uses Clustering to Discover Candidate Structure
Genomics, ecology, astronomy and many other fields use clustering to identify patterns in high-dimensional measurements.
The clusters can reveal candidate subtypes or regimes worth further study.
But discovery requires validation. A statistical cluster is evidence of structure under a method, not automatic proof of a new natural kind.
When Clustering Is Exactly the Right Loss
Human beings cannot inspect millions of individual points one by one.
A good clustering can expose broad organisation, reduce search space, support sampling and reveal anomalous cases.
It is powerful because it makes a large world navigable.
When Clustering Becomes Dangerous
- The algorithm is treated as discovering one true grouping.
- Scaling or distance choices are hidden.
- Projection artefacts are mistaken for source-space clusters.
- Outliers are forced into groups.
- Cluster instability is ignored.
- Group averages are applied to individuals.
- Minority structures disappear inside dominant clusters.
A Practical Clustering Audit
- Similarity: what distance or relationship defines neighbourhood?
- Scale: how were features normalised?
- Algorithm: what shapes of cluster can the method represent?
- Parameters: how sensitive are results to k, density or linkage choices?
- Stability: do groups survive resampling and perturbation?
- Uncertainty: are borderline cases visibly borderline?
- Receiver: is the cluster for exploration, prediction, intervention or reporting?
Sources and Further Reading
- scikit-learn User Guide: Clustering for practical distinctions among k-means, hierarchical, density-based and related methods.
- Foundational literature on cluster validation, silhouette analysis and stability.
Continue Through eduKateSG
Continue with How Lossy Works | Projection, How Lossy Works | Feature Selection, and How Lossy Works | Binning.
Final Thought: A Group Is a Lens, Not a Destiny
Clustering gives civilisation a way to see structure without carrying every individual detail at once.
Its wisdom lies in remembering that the label is smaller than the person, point or object it describes.