Run the embedding once.
Three beautiful clusters appear.
Change one parameter.
One cluster splits.
Change the random seed.
Two islands trade places.
The dataset did not change.
Projection stability asks whether the structure we are about to believe belongs to the source—or only to one reasonable way of projecting it.
This is the third pillar beneath How Lossy Works | Projection. The master owns dimensional loss. This article owns robustness: whether a visual conclusion survives reasonable changes in preprocessing, parameters, random state, samples and methods.
Quick Read
A stable projection preserves the important qualitative claim under reasonable perturbations to the analysis. Stability does not require every point to remain in the same pixel. It asks whether the substantive structure—neighbour relations, separation, ordering, trajectory or decision boundary—survives. PCA is deterministic under many solver/settings combinations but remains sensitive to preprocessing and sampling. t-SNE uses a non-convex objective and can yield different layouts under different initialisations; its perplexity changes the effective neighbourhood scale. UMAP’s n_neighbors and min_dist parameters explicitly change the local-versus-global emphasis and packing of the embedding. Important claims should therefore be tested across a declared stability envelope.
projection claim → vary seed / parameter / preprocessing / sample / method → compare invariant structure → stable claim or projection-specific claim
Stability Is About the Claim, Not the Picture
Two UMAP plots can be rotated versions of one another.
The coordinates changed.
The neighbourhood structure may not have.
Conversely, two plots can look broadly similar while a small subgroup repeatedly jumps between clusters.
That subgroup may be exactly where the scientific question lives.
Do not ask only “does the plot look similar?” Ask “did the relationship I care about survive?”
t-SNE Makes Instability Explicit
Current scikit-learn documentation states that t-SNE’s cost function is non-convex, so different initialisations can produce different results. It also recommends setting a random state for reproducibility when appropriate.
This means one visually persuasive t-SNE run should never be treated as if the optimiser had discovered the only valid two-dimensional geography of the dataset.
Reproducibility and Stability Are Different
Fix the random seed and you can reproduce the same result.
That proves execution reproducibility.
It does not prove the result is stable to other reasonable seeds.
A frozen fragile projection is still fragile.
Perplexity Changes the Neighbourhood Scale in t-SNE
Scikit-learn describes perplexity as related to the number of nearest neighbours considered by the algorithm and notes that different values can produce significantly different results.
Its own example warns that cluster size, distance and shape can vary with initialisation and perplexity and do not always carry literal meaning.
So a cluster that exists only at one narrow perplexity may be a scale-dependent representation rather than a robust natural category.
Learning Rate Can Reshape the Optimisation Landscape
Scikit-learn notes that a t-SNE learning rate that is too high can make the embedding look like a ball, while a value that is too low can compress points into a dense cloud with few outliers.
The message is larger than one parameter:
visual geometry can be partly algorithmic geometry.
UMAP Has an Explicit Local–Global Dial
UMAP’s n_neighbors controls the scale of neighbourhood structure used in constructing the manifold approximation. Smaller values emphasise very local structure; larger values allow broader structure to influence the embedding.
The same data can therefore look more fragmented or more globally organised depending on the scale at which neighbourhood is defined.
UMAP min_dist Changes How Tightly Points Pack
Official UMAP documentation explains that min_dist controls how tightly points may pack in the low-dimensional representation. Lower values can produce tighter clumps; higher values produce more spread-out embeddings and can emphasise broader structure.
A dense-looking cluster can therefore be partly the consequence of a packing parameter.
Recent Reviews Treat Stability as a First-Class Limitation
A recent comprehensive review of dimensionality-reduction methods highlights instability, overfitting and poor generalisation as significant reliability problems for nonlinear and graph-based embeddings, especially when stochastic optimisation and hyperparameter sensitivity affect downstream interpretation.
The practical implication is not to reject these methods.
It is to stop treating one run as ontology.
PCA Can Be Numerically Reproducible and Scientifically Unstable
PCA often feels safer because it is linear and, for a fixed dataset and deterministic solver, highly reproducible.
But change:
- feature scaling;
- which observations are included;
- how missing values are handled;
- outlier treatment;
- the measurement units;
and the principal directions can change substantially.
Deterministic does not mean preprocessing-independent.
Near-Tied Components Can Rotate Easily
If two principal components have very similar explained variance, small perturbations in the sample can rotate the estimated basis within that near-equal subspace.
Individual component loadings may look unstable even when the broader subspace is relatively stable.
The right invariant may therefore be the span of several components rather than one named axis.
Sampling Stability Matters
Remove 5% of the observations.
Bootstrap the dataset.
Repeat the embedding.
If the whole interpretation collapses, the original projection may have depended heavily on particular sample points.
Outliers Can Anchor or Distort the View
A handful of extreme observations can dominate variance in PCA.
Rare isolated points can affect local neighbourhood graphs.
Before removing them, ask whether they are:
- measurement errors;
- legitimate rare cases;
- the very phenomenon of interest.
Stability analysis should not quietly delete inconvenient reality.
Feature Scaling Is a Stability Dimension
Standardise every feature.
Now do not.
If a claimed structure appears only under one scaling choice, investigate whether that scaling matches the meaning of the variables.
Normalisation owns the general scale transformation. Projection Stability asks whether the interpretation survives defensible alternatives.
Missing-Data Treatment Can Rearrange the Projection
Delete incomplete rows.
Mean-impute them.
Model-impute them.
Each choice changes the geometry before projection begins.
Imputation owns the missing-data replacement problem.
Feature Selection Can Create False Stability
Choose the features after seeing which ones produce a visually pleasing separation.
Then rerun the projection.
The structure looks wonderfully stable because selection already encoded the desired result.
Stability testing must include upstream analytical choices, not only the final embedding seed.
Feature Selection owns that upstream loss.
Method Stability Is Stronger Than Parameter Stability
A structure survives several t-SNE perplexities.
Good.
Does it survive UMAP?
PCA?
A direct high-dimensional distance check?
If yes, the claim has stronger evidence than one method can provide alone.
But Different Methods Preserve Different Things
Failure to look identical across methods is not automatically a problem.
PCA prioritises variance under a linear subspace.
t-SNE emphasises neighbourhood similarity.
UMAP makes its own neighbourhood/topology trade-offs.
The fourth pillar, Projection Objectives, owns why “same picture” is not the right universal expectation.
Topology Can Be More Stable Than Coordinates
Clusters rotate.
Distances stretch.
But nearest-neighbour relationships remain similar.
If the scientific question concerns neighbourhood membership, coordinate drift may be acceptable.
Cluster Labels Need Their Own Stability Test
Do not infer stable categories just because coloured points remain separated visually.
Ask whether membership changes under:
- resampling;
- projection parameters;
- clustering algorithm;
- distance metric;
- feature preprocessing.
The category should earn stability separately from the picture.
Trajectories Need Stability Too
A projected manifold appears to show a path from state A to B to C.
If the path reverses, branches or disappears under reasonable settings, interpret developmental or causal stories cautiously.
Visual continuity can be a projection artefact just as visual clustering can.
Stability Should Be Localised
Most of the embedding is stable.
One boundary region is not.
Do not label the whole plot “unstable.”
Identify where uncertainty concentrates.
This turns instability into useful information about ambiguous cases.
Ensembles Can Reveal Consensus Structure
Run multiple seeds or parameter settings.
Align comparable embeddings where appropriate.
Look for relationships that persist.
Recent dimensionality-reduction reviews discuss ensemble and multi-scale approaches as ways to mitigate the fragility of relying on one embedding.
A Stability Envelope Is Better Than One Magic Setting
Define a range of reasonable settings before interpreting:
- perplexity 20–50;
- several random seeds;
- UMAP n_neighbors across meaningful scales;
- alternative min_dist values;
- with and without standardisation where scientifically defensible.
If the conclusion survives the envelope, confidence rises.
If it exists only at one cherry-picked point, the claim should be described as parameter-specific.
Do Not Tune for the Prettiest Story
Humans prefer separation.
Clear islands feel explanatory.
If parameters are chosen because the resulting picture looks most convincing, visual aesthetics become an undeclared optimisation objective.
Parameter selection should be tied to method logic, validation or downstream task—not beauty alone.
Stability Is Not Truth
A wrong preprocessing pipeline can produce the same wrong projection every time.
A biased sample can yield perfectly stable clusters.
Stability answers:
does this result depend heavily on arbitrary analytical perturbation?
It does not answer:
is the source data valid and the interpretation causally correct?
World Return Is the Final Stability Test
A cluster appears stable across seeds and methods.
Now inspect its members.
Do they share the claimed real-world property?
Does the projection predict new observations?
Representation robustness is strongest when it returns successfully to the richer world that produced the data.
A Better Projection-Stability Model
claim → define reasonable perturbations → repeated projections → align comparable structure → measure invariant relations → localise unstable regions → validate in source space → publish claim with stability boundary
A 30-Lens Projection Stability Audit
- Claim: what exact conclusion is being drawn?
- Method: PCA, t-SNE, UMAP or another projection?
- Objective: what structure is the method designed to preserve?
- Randomness: is optimisation stochastic?
- Seed: was random state recorded?
- Seed sweep: does the claim survive several seeds?
- Perplexity: for t-SNE, what neighbourhood scales were tested?
- Learning rate: does a reasonable range preserve the claim?
- n_neighbors: for UMAP, is structure local-scale dependent?
- min_dist: is cluster compactness parameter-driven?
- Metric: what notion of distance is used?
- Scaling: were features standardised?
- Units: do raw scales dominate geometry?
- Missing data: how was imputation handled?
- Feature selection: was upstream selection fixed independently?
- Sample: does resampling preserve the claim?
- Outliers: do rare points anchor the geometry?
- Component tie: are PCA axes near-degenerate?
- Coordinate invariance: is rotation/translation irrelevant to the claim?
- Neighbour stability: do local relationships persist?
- Cluster membership: who changes groups?
- Trajectory: do paths persist?
- Subgroup: where is instability concentrated?
- Method comparison: does another projection support the same relationship?
- High-dimensional check: is the relation present before projection?
- Parameter selection: was a visually pleasing result cherry-picked?
- Envelope: what range counts as reasonable?
- Reproducibility: can the exact run be recreated?
- World validation: does the stable structure correspond to external evidence?
- Disclosure: is instability communicated where it matters?
Laboratory 1: Seed Sweep
Run a stochastic embedding several times with different seeds. Ignore global rotation and ask which point neighbourhoods and cluster memberships remain stable.
Laboratory 2: Perplexity Ladder
Compare t-SNE across several reasonable perplexities. Record which claims survive and which appear only at one scale.
Laboratory 3: Remove Ten Percent
Resample the dataset repeatedly. If a supposed cluster vanishes whenever a few specific observations disappear, describe it as sample-sensitive rather than fixed structure.
For Primary Readers
Build the same block tower three times from slightly different starting positions. Ask which features stay recognisably the same and which depend on setup.
For Secondary Readers
Define one visual claim, vary one projection setting at a time, and record whether the claim survives rather than whether every dot stays fixed.
For Advanced Readers
Model stability as robustness of an invariant or downstream functional under perturbations to data, preprocessing, optimisation and hyperparameters. Coordinate reproducibility is weaker evidence than persistence of task-relevant topology or neighbourhood structure.
Common Misconceptions
- “If I set a seed, the projection is stable.” A seed makes one stochastic run reproducible; stability requires perturbation testing.
- “PCA is stable because it is linear.” It can still be sensitive to scaling, sampling and near-tied components.
- “Different UMAP layouts mean the method failed.” Some coordinate changes are harmless; evaluate the relationship relevant to the task.
- “The prettiest separation is the best parameter choice.” Visual attractiveness is not a scientific objective unless explicitly justified.
- “Stable means true.” Biased data or wrong assumptions can produce stable error.
Research Corridor
- scikit-learn — t-SNE — non-convex optimisation, random state and parameter behaviour.
- scikit-learn — Effect of t-SNE Perplexity — examples showing cluster size, distance and shape can vary.
- UMAP — Basic Parameters — n_neighbors and min_dist trade-offs.
- Comprehensive Review of Dimensionality Reduction Algorithms — stability, generalisation and parameter sensitivity.
Frequently Asked Questions
What does projection stability mean?
It means that the task-relevant structure or conclusion survives reasonable changes in random state, parameters, preprocessing, sampling or projection method.
Why can t-SNE plots change between runs?
t-SNE optimises a non-convex objective, so different initialisations can reach different local solutions; parameters such as perplexity also change the effective neighbourhood scale.
Should every stable claim survive every projection method?
No. Methods preserve different structures. The right comparison asks whether the claimed source-space relationship is supported by methods whose objectives are relevant to that claim.
Final Thought: A Strong Pattern Survives More Than One Good-Looking Plot
Projection turns complex reality into a visible argument.
Before trusting that argument, move the lamp slightly and see whether the important shadow is still there.
PROJECTION · FOUR PILLAR LEGS
Return to Projection, or continue through Projection Direction, Projection Collisions and Projection Objectives. Return to the Information & Representation Hub.