How Superposition in Neural Representations Works | Why Neural Networks Can Store More Features Than They Have Dimensions

x = a₁d₁ + a₂d₂,

d₁·x = a₁||d₁||² + a₂(d₁·d₂).

21. The residual stream turns superposition into a shared communication medium

Transformers repeatedly add information into a residual stream. Attention heads and MLPs read from that stream, compute something, and write back into it.

This additive architecture encourages us to think in terms of directions. A component can write a feature-like vector into the stream. Later components can read projections or nonlinear functions of that state.

If many useful features share the same residual dimensions, the stream behaves like a crowded communication bus. Different messages coexist as vector components.

The benefit is compositional flexibility. Many components can contribute simultaneously without requiring a separate wire for every possible concept.

The cost is interpretability. A coordinate does not belong to one feature. Reading one neuron can mix several signals written by different components.

Superposition therefore helps explain why activation patching and direction-based interventions often work more naturally than neuron-by-neuron stories. The functional object may be a direction or subspace rather than an axis.

22. Attention heads can read and write features without owning them

An attention head is sometimes described as if it contains a concept.

A more careful view is that attention heads implement transformations on representations. They can detect patterns through queries and keys, aggregate values, and write vectors into the residual stream.

If the stream is superposed, a head can interact with a feature direction without that feature being localized inside the head as one monosemantic unit.

Likewise, one head can participate in several behaviours depending on context because its input contains different active feature combinations.

This makes circuit interpretation relational. We ask which directions are read, what transformation occurs, which directions are written, and under what contexts the path matters.

The superposition hypothesis does not eliminate circuit analysis. It changes the units from “this neuron is the circuit” toward “this feature direction is carried through components that read and write overlapping spaces.”

23. MLP layers can behave like feature detectors and feature writers at the same time

A transformer MLP usually expands the residual state into a wider hidden layer, applies a nonlinearity, then projects back down.

The hidden neurons can look more feature-like because the activation function creates coordinate-specific gating.

But even MLP neurons can be polysemantic.

One way to understand the layer is as a bank of conditional transformations. A hidden unit activates when some combination of residual features crosses its input threshold and then writes a direction back into the residual stream.

If input features are superposed, the neuron’s preactivation mixes them. If the hidden neuron itself serves several contexts, its activation becomes polysemantic. Its output direction can also contribute to several downstream feature combinations.

Sparse-autoencoder work has often targeted MLP activations because the hidden layer is a convenient place to inspect this mixture, but the general decomposition problem can apply to other activation sites as well.

24. More model width changes the pressure for superposition, but does not make it disappear automatically

A wider layer provides more independent dimensions.

That can reduce the need to pack features aggressively.

But larger models often learn more features too.

Capacity demand can grow with capacity supply.

A model with twice the width may not become twice as monosemantic if optimization uses the extra dimensions to represent a richer set of rare features, more contexts, finer distinctions and additional intermediate computations.

This is one reason sparse-autoencoder dictionaries often expand far beyond the model’s activation dimension. Researchers continue to find additional candidate features as dictionary width grows.

We should therefore distinguish architectural width from feature count. Scaling the first does not put a known ceiling on the second.

25. Data scale can sharpen or multiply the features a model learns

Training on more data exposes a model to more recurring structures.

Common patterns can become cleaner because the model receives more evidence.

Rare patterns can become worth representing because they appear often enough in an enormous corpus.

These effects push in different directions for interpretability.

More data can produce sharper feature structure.

It can also increase the total number of useful features competing for finite dimensions.

Towards Monosemanticity reported that training sparse autoencoders on more activation data produced subjectively sharper features. That observation concerns the post-hoc dictionary, not merely the base model.

The broader principle is that representation resolution depends on both model capacity and the amount of structure available to learn.

26. Feature density is one of the most useful empirical quantities to inspect

Feature density asks how often a candidate feature is active.

A feature active on nearly every token behaves differently from one active on one in a million.

Extremely dense features are expensive to superpose because they interfere often.

Extremely rare features may be difficult to learn reliably and difficult to validate statistically.

A dictionary containing many dead features can look impressively wide while using little of that width.

Density distributions therefore reveal whether an interpretation method is actually allocating its feature budget.

For superposition, density also predicts interference pressure. Features that activate frequently need cleaner geometry or stronger downstream separation mechanisms than features that almost never overlap.

27. Dead dictionary features are wasted interpretability capacity

When a sparse autoencoder is made very wide, some latent units may never activate.

These dead features consume parameters without contributing to reconstruction or interpretation.

Early Anthropic work used feature resampling: periodically identify dead units and reset them toward activation examples the current dictionary reconstructs poorly.

This illustrates a subtle point. Overcompleteness alone does not recover superposed structure.

The optimization must actually spread useful work across the dictionary.

Later sparse-autoencoder techniques use different activation functions, sparsity penalties, thresholding rules and training procedures to manage similar problems.

The specific recipe can change. The diagnostic remains: how much of the learned feature vocabulary is alive and functionally useful?

28. Reconstruction error measures what the dictionary fails to explain

A sparse decomposition is only useful if it can reconstruct the original activation accurately enough for the scientific question.

Let x be a model activation and \hat{x} the reconstruction from learned features.

A common reconstruction term is mean squared error:

L_rec = ||x − x̂||².

Low error means the dictionary captures much of the activation variation.

But low error alone does not guarantee interpretability.

A dense autoencoder can reconstruct perfectly while producing latents that are just another rotated basis.

Sparsity pushes the model toward using a small subset of latent features for each activation.

The interpretability objective is therefore multi-part: reconstruct enough, activate sparsely enough, and produce features that correspond to stable and functionally meaningful patterns.

29. Sparse autoencoders live on a sparsity–reconstruction frontier

Increase the sparsity penalty and fewer features activate.

Push too far and reconstruction worsens because the dictionary cannot express enough of the activation.

Reduce the sparsity pressure and reconstruction improves, but features can become dense and less interpretable.

This creates a Pareto frontier rather than one magic score.

Comparing sparse autoencoders requires matching sparsity and reconstruction fairly. A model that looks better only because it uses twice as many active latents per token may not be a better decomposition.

For superposition research, the frontier matters because a decomposition that leaves substantial residual error may simply fail to recover important superposed features.

30. Reconstruction is weaker than explaining behaviour

Two decompositions can reconstruct activations equally well and differ radically in interpretability.

The stronger question is whether the learned features explain model behaviour.

Does a feature’s activation predict when a behaviour occurs?

Does intervening on the feature cause the predicted output change?

Can downstream weights or attribution paths explain how the feature affects logits or decisions?

Can the feature distinguish two behaviours that raw neurons mix together?

This moves the evaluation from “the dictionary compresses activations” to “the dictionary exposes a useful mechanism.”

31. Steering is a powerful but imperfect validation test

Feature steering adds a learned feature direction to the model’s activation during generation.

If a candidate feature is labelled “Golden Gate Bridge,” increasing it may make the model mention the bridge or related San Francisco concepts more often.

This supports the claim that the direction has causal influence.

It does not show that the feature is used in exactly that form during ordinary computation.

Large interventions can push the model off-distribution. A direction can have broad downstream effects because it overlaps with other features. Steering can activate correlated circuitry that the natural feature does not usually control alone.

Good validation therefore compares several intervention strengths, checks specificity, uses negative controls and tests whether the effect occurs on prompts where the feature should matter.

32. Ablation asks whether the model needs the feature, not merely whether it can be pushed

Steering tests sufficiency-like influence: if we add the feature, can we cause behaviour?

Ablation tests a necessity-like question: if we remove or suppress the feature, does the behaviour weaken?

Neither test alone proves complete necessity or sufficiency because neural networks contain redundancy and alternative pathways.

A behaviour can survive ablation because another feature compensates.

Adding a feature can cause a behaviour the model would never naturally produce from that feature alone.

Using both tests, plus attribution and held-out prediction, creates a stronger evidence package.

33. Superposed features can interact nonlinearly downstream

Feature A and feature B can share activation dimensions and still participate in different nonlinear circuits.

A downstream MLP may fire only when both are active.

An attention head may use A as a query-like condition and B as a content signal.

A later layer may rotate their combination into a cleaner direction.

This means a feature dictionary is not a complete circuit diagram.

It supplies candidate state variables. Mechanistic interpretation still needs to understand how those variables are transformed and combined.

The distinction mirrors the difference between vocabulary and grammar. Identifying the words does not explain the sentence.

34. Composition and superposition can coexist

A model may represent “red” and “car” as features that compose to represent a red car.

Those same feature directions can be superposed with many other sparse features in the finite activation space.

Composition describes how active features combine to express richer states.

Superposition describes how the vocabulary of possible features shares representational dimensions.

Confusing the two leads to a common mistake: assuming that because one representation is distributed across neurons, every neuron must be polysemantic because of superposition. Distribution can support clean composition in a high-dimensional space without severe feature crowding.

The empirical question is how many features, with what sparsity and overlap, are represented relative to the available dimensions.

35. Superposition can be conditional on context

A network may reuse representational directions differently across contexts.

One feature family can dominate in code.

Another can dominate in poetry.

If the contexts rarely overlap, the model can reuse capacity with limited interference.

This is more flexible than a single global dictionary where every feature competes equally with every other feature.

Contextual gating, attention and layer normalization can alter which directions are active or readable.

Therefore an interpretability method trained on a broad activation mixture may merge context-specific features that would separate under conditional analysis.

36. Token position can create another axis of feature reuse

Transformers process sequences, and the same neuron is evaluated at every token position.

A feature can be meaningful only near the beginning of a sequence, inside quoted text, after a colon, or while the model is producing code.

The architectural neuron is reused across positions by design.

Superposition can add another layer of reuse across semantic features.

Feature interpretation should therefore preserve position and context. Top-activating tokens without surrounding text can make a feature look incoherent because the trigger depends on a relation to earlier tokens.

A good feature browser shows context windows, not isolated token strings.

37. A feature can encode a process state rather than an object or topic

Interpretable features are often labelled with nouns: DNA, law, French, poetry.

But models also need process features.

“Inside a quotation.”

“Currently listing items.”

“Need to close an HTML tag.”

“Carrying a subject–verb dependency.”

These state-like features can be sparse and superposed just like semantic content features.

They can also form finite-state-machine-like circuits where one feature activates another in sequence. Towards Monosemanticity described motifs suggestive of such structured feature transitions.

This broadens the interpretability target from “what concept is represented?” to “what computational state is the model in?”

38. Superposition can support memory-like storage without literal slots

It is tempting to imagine model knowledge as a database of dedicated memory cells.

Neural networks generally do not work that way.

Knowledge is distributed through weights and activations. A fact can be reconstructed through interactions among many features and circuits.

Superposition adds capacity to intermediate states by allowing many candidate variables to share dimensions.

This does not mean the model “stores two facts in one neuron” in a simple digital sense.

It means the activation coordinate participates in multiple learned directions whose significance emerges only when the full vector and downstream computation are considered.

39. Editing one feature can disturb neighbours because the geometry is shared

If a feature direction is superposed with others, interventions are not guaranteed to be perfectly selective.

Adding a direction can change projections onto neighbouring features.

A sparse-autoencoder latent may provide a more selective direction than a raw neuron, but the decoder features need not be orthogonal.

Steering one latent can therefore influence correlated features or downstream circuits.

The correct evaluation measures specificity:

• target behaviour changes,
• unrelated behaviours stay stable,
• neighbour features are monitored,
• effects scale smoothly with intervention strength,
• the intervention works in contexts where the feature naturally matters.

Interpretability should not confuse causal leverage with surgical precision.

40. Feature collisions are easiest to notice when two rare signals appear together

A superposed representation can perform perfectly on inputs where only one of two overlapping features is active.

The failure appears when both occur together.

This suggests a diagnostic strategy: construct collision tests.

If one feature detects legal citations and another detects chemical formulas, find prompts containing both.

Does each feature remain readable?

Does the model confuse them?

Does the sparse-autoencoder decomposition activate both latents cleanly or produce a third mixed latent?

Co-activation challenge sets are particularly valuable because random corpus examples may rarely expose the interference regime that motivated superposition in the first place.

41. Adversarial mixtures test whether a feature label survives outside its favourite examples

A feature can look monosemantic on its top 100 activations and fail on carefully chosen counterexamples.

Suppose a feature is labelled “medical diagnosis.”

Test medical text with no diagnosis.

Test diagnosis language in veterinary, legal or fictional settings.

Test syntactically similar text with different semantics.

Test negated diagnoses.

Test the same topic in another language.

These counterexamples reveal whether the feature tracks the intended concept, a proxy, a lexical cue or a mixture.

42. Automated feature labels are hypotheses, not ground truth

Large sparse dictionaries can contain millions of features. Humans cannot inspect all of them manually.

Researchers therefore use language models to summarize top-activating examples and generate feature descriptions.

This scales inspection but introduces a new model into the measurement chain.

The labeler can hallucinate a coherent story from noisy examples.

It can choose an overly broad label.

It can miss a hidden trigger shared across examples.

Automated interpretability scores should therefore be calibrated against human judgement and causal tests. A high score means the labeler found a predictable description under its evaluation protocol, not that the feature has been fully explained.

43. The explanation gap: a feature can be interpretable without explaining the model’s decision

Suppose we find a perfectly interpretable “fraud” feature.

The model makes a suspicious financial decision.

Did the fraud feature cause the decision?

Not necessarily.

The feature may activate without being read downstream.

Another correlated feature may drive the output.

The decision may depend on a nonlinear combination not captured by the label.

This limitation is visible in current interpretability research. In 2026, Anthropic reported CHIVE evaluations where several activation-reading tools, including sparse autoencoders, did not improve prediction of counterfactual behavioural changes over a transcript-only baseline for naturally occurring behaviours. That result does not show SAEs are useless. It shows interpretable features do not automatically close the explanation gap.

44. The 2026 frontier is shifting from finding features to validating explanations

Early mechanistic interpretability often celebrated finding a neuron or feature with a compelling semantic label.

The frontier increasingly asks harder questions:

• Does the feature generalize across contexts?
• Does it predict behaviour on held-out examples?
• Does causal intervention change the predicted behaviour?
• Does the explanation outperform simpler behavioural baselines?
• Does the feature participate in a reproducible circuit?
• Does the decomposition remain stable after model compression, fine-tuning or distribution shift?

Recent work on pruning and sparse-autoencoder robustness, scientific-model SAEs and counterfactual evaluation reflects this shift. The important question is no longer only “can we name a latent?” but “does the latent support reliable mechanistic prediction?”

45. Model pruning can distort a learned feature dictionary even when model behaviour degrades only modestly

A sparse autoencoder is trained against one activation distribution.

Prune the base model and that distribution changes.

Recent 2026 work studying pruning and SAE robustness argues that activation-aware pruning methods can preserve SAE behaviour better than simple magnitude pruning because they better respect activation geometry.

This matters for superposition because interpretability tools are not detached observers. They are fitted to a particular representational geometry.

Change the base model and the feature dictionary may need to be retrained or revalidated.

A deployment pipeline that quantizes, prunes or fine-tunes a model after interpretability analysis should not assume the old feature atlas remains valid.

46. Fine-tuning can move, split or repurpose features

A feature atlas built for a base model is not guaranteed to remain valid after instruction tuning, reinforcement learning, domain adaptation or safety fine-tuning.

Fine-tuning changes weights.

Changed weights alter activation distributions.

Feature directions can rotate, feature densities can change, old features can split and new features can appear.

Some representations may remain approximately aligned because fine-tuning is small relative to pretraining. Others can change sharply in the layers most affected by the objective.

This suggests a versioning discipline for interpretability:

• base-model version,
• fine-tune version,
• activation site,
• dictionary training data,
• dictionary width and sparsity regime,
• validation suite version.

Without those coordinates, a feature identifier such as “feature 30277” is not portable scientific meaning.

47. Matching features across models is harder than matching labels

Suppose two independently trained language models each have a sparse-autoencoder feature labelled “Python list comprehension.”

Are they the same feature?

A semantic label is suggestive but insufficient.

One can compare top-activating examples, activation patterns over a shared dataset, decoder directions after alignment, downstream effects, causal steering and the circuits that read the feature.

Even then, the correspondence can be many-to-one. A broad feature in one model may split into three specialized features in another. Two narrow features may merge into one broader latent.

Cross-model feature matching is therefore a representational correspondence problem, not a naming problem.

48. A feature atlas is a map, not the territory

Large interpretability projects increasingly produce browsable feature atlases.

For each feature, the atlas may show top activating examples, activation statistics, automated descriptions, downstream weights and steering demonstrations.

This is useful infrastructure.

It can also create false confidence because the interface makes every feature look like a named object.

The atlas omits what the feature fails to capture, which examples nearly activate it, which other features overlap, how stable it is across model versions and whether the human label predicts behaviour.

A good atlas should therefore expose uncertainty and diagnostics, not only attractive examples.

49. Large-scale sparse autoencoders show both promise and the scale of the problem

OpenAI’s 2024 work Extracting Concepts from GPT-4 trained sparse autoencoders with up to millions of features on model activations and reported many human-interpretable learned features.

The same work emphasized unresolved limitations. Many features remained difficult to interpret, interpretations could be spurious, the autoencoder did not capture all of the model’s behaviour, and scaling toward full concept coverage could require extremely large dictionaries.

This is exactly the superposition tension at scale.

If the model uses a huge number of sparse features, an interpreter needs a dictionary large enough to recover them without excessive merging.

But a larger dictionary increases training cost, dead-feature risk, feature splitting, validation burden and the difficulty of deciding which resolution is scientifically useful.

50. Scaling Monosemanticity moved the question into a frontier-scale model

Anthropic’s 2024 Scaling Monosemanticity applied sparse-autoencoder methods to Claude 3 Sonnet and reported millions of candidate features at a chosen activation site.

The importance is not that every feature suddenly became understandable.

The importance is that a decomposition approach motivated by superposition could scale far beyond toy networks and one-layer transformers.

The work also exposed new questions: how many features are needed, how feature quality changes with scale, how to evaluate millions of latents, how to distinguish useful semantic structure from dataset artefacts, and how to connect features into causal circuits.

Scaling the dictionary solves one bottleneck and creates several others.

51. There is no reason every scientific question should use the same feature resolution

Suppose a small dictionary has one feature labelled “biology.”

A larger dictionary splits it into genetics, ecology, cell biology and evolution.

A still larger dictionary splits genetics into DNA sequencing, Mendelian inheritance, CRISPR and population genetics.

Which representation is correct?

All may be useful at different scales.

A safety audit looking for biological weapon content may prefer coarse categories.

A circuit analysis of gene-editing reasoning may need the fine features.

Feature dictionaries should therefore be evaluated against the granularity of the scientific job rather than one universal ideal of monosemanticity.

52. Feature merging is the mirror image of feature splitting

A dictionary can be too small to separate nearby concepts.

One latent may activate for both “violin” and “cello.”

That can be interpreted as one broad “bowed strings” feature or as a merged feature caused by insufficient dictionary resolution.

How can we tell?

Increase dictionary capacity and see whether the feature splits reproducibly.

Test whether violin and cello have different downstream effects.

Look for examples requiring one distinction but not the other.

A merged feature is not necessarily bad. It may match the level of abstraction used by the model at that layer. But one should not mistake dictionary bottlenecks for model ontology.

53. Different layers can represent the same concept at different resolutions

Early layers often track local lexical or perceptual patterns.

Middle layers can combine them into contextual concepts.

Later layers can represent task-relevant, prediction-relevant or decision-relevant states.

A “Paris” feature in an early layer may reflect the token or spelling.

In a later layer it may reflect the city as a semantic entity, travel context, France, or the expected answer to a question.

Superposition therefore changes across depth because the feature vocabulary changes.

An interpretability project should not assume one layer is “the” representation of a concept. It should choose the activation site based on the computational question.

54. Local features and global state features can share the same layer

One feature can respond to a local token pattern such as “http://”.

Another can represent a global state such as “the model is writing a citation.”

Both can be active at the same token.

Superposition can therefore mix different semantic scales.

This is one reason top-activating examples can look heterogeneous. A feature might be triggered by a structural state expressed through many surface forms.

Interpretation needs to ask whether examples share a word, a topic, a syntax, a task state, a prediction target or a downstream effect.

55. A feature direction is not the same thing as a trained probe

A linear probe can often decode a concept from a representation.

That does not prove the probe direction is a native feature used by the model.

The probe can combine many distributed signals that the model never reads with that exact weight vector.

A sparse-autoencoder feature is learned without the external concept label, which gives it a different evidential status.

But the SAE feature is still a post-hoc learned direction and not automatically a privileged computational object.

Probe decodability, dictionary discovery and causal use are three different questions.

56. Linear decodability can survive heavy superposition

A representation can be superposed and still allow a linear decoder to recover one feature well.

The relevant feature direction may remain separable despite sharing coordinates.

This is why “information is linearly decodable” does not imply “there is one neuron for it.”

Likewise, a feature can be hard to decode linearly and still matter through a nonlinear circuit.

Superposition therefore separates three levels:

• information present in the activation,
• information accessible to a chosen decoder,
• information actually used by the model’s downstream computation.

Interpretability claims should say which level they establish.

57. Some meaningful features may be nonlinear objects, not single directions

The simplest superposition picture treats each feature as a direction.

Real neural representations can be more complicated.

A feature may occupy a curved manifold.

Its activation may depend on an interaction between two directions.

It may be represented differently across contexts.

A linear sparse autoencoder approximates such structure with multiple dictionary elements or fails to capture it fully.

This is another reason not to equate “feature” with “one vector” universally. Vector features are a powerful first-order approximation for many mechanistic questions, not a theorem that every internal variable is linear.

58. A curved concept can look like many features when approximated linearly

Imagine a semantic variable that changes smoothly across context.

In one context its local direction points northeast.

In another it points north.

A linear dictionary may need several features to tile this curved structure.

Researchers can then observe “feature splitting” where the deeper reality is curvature or conditional geometry rather than several independent concepts.

This possibility connects superposition to Manifold and to the later Neural Population Geometry owner, while keeping the scientific objects distinct.

59. Superposition matters for AI safety because dangerous features need not have dedicated neurons

If a safety-relevant concept were always represented by one monosemantic neuron, monitoring would be easy.

Read the neuron.

Set a threshold.

Intervene when it fires.

Superposition breaks that simple architecture of trust.

A dangerous planning feature may be spread across many neurons and share them with benign concepts.

A raw-neuron threshold can miss the feature or produce false alarms.

Dictionary methods can provide candidate safety features, but they introduce their own recall and validation problems. A safety monitor built on SAEs must measure feature coverage, false negatives, drift and adversarial evasion.

60. An adversary can target the interpreter as well as the model

Suppose a safety system blocks output whenever a learned “malware intent” feature activates strongly.

An adversarial prompt may seek a semantically equivalent route that activates a different feature combination.

If the dictionary has incomplete coverage, the model can express the dangerous computation through residual directions the interpreter does not monitor.

If the feature is broad, an attacker may distribute the behaviour across several weaker activations below threshold.

This does not make mechanistic monitoring hopeless. It changes the adversarial model.

One should test the monitor against paraphrases, obfuscation, multilingual inputs, indirect planning and feature-splitting strategies rather than only standard prompts.

61. Superposition is a form of representational compression with structured loss

A model with F potential features and d dimensions, where F is much larger than d, cannot preserve every feature independently in a linear orthogonal code.

Superposition compresses the feature vocabulary into a smaller space.

The compression is not arbitrary.

Sparse features can share capacity because interference is paid only when they overlap.

Important features can receive cleaner geometry.

Less important features can tolerate more error.

This is why superposition belongs near the conceptual territory of compression but does not replace the existing How Compression Works owner. The present article explains one learned neural capacity mechanism rather than compression in general.

62. Superposition can be lossy without being useless

Overlapping feature directions create noise.

If the downstream computation tolerates that noise, the code remains useful.

This resembles lossy compression: some precision is surrendered to gain capacity.

The crucial question is whether the lost precision affects important decisions.

A feature used only for broad topical routing may tolerate interference.

A feature controlling a precise arithmetic carry operation may require cleaner representation.

Optimization can allocate capacity accordingly because errors on high-impact features produce larger loss.

63. Robust features survive interference, distribution shift and nearby activations

A feature is more scientifically convincing when it can be identified across many natural contexts.

It should activate for diverse examples sharing the hypothesized property.

It should stay quiet on carefully matched negatives.

It should remain detectable when neighbouring features co-activate.

It should preserve its causal role under modest distribution shifts.

These tests are more demanding than inspecting top activations, but they are precisely what superposition makes necessary.

64. An evidence ladder for a claimed superposed feature

1. Activation pattern: the feature fires on examples sharing a proposed property.
2. Matched negatives: similar examples lacking the property do not trigger it strongly.
3. Automated prediction: the label predicts activation on held-out examples.
4. Cross-run stability: similar features recur across dictionary seeds or widths.
5. Cross-model recurrence: analogous features appear in independently trained models where theoretically expected.
6. Downstream association: weights or attribution paths connect the feature to plausible outputs.
7. Steering: adding the feature changes behaviour in the predicted direction.
8. Ablation: reducing the feature weakens predicted behaviours where the model uses it.
9. Specificity: unrelated behaviours are not broadly disturbed.
10. Counterfactual prediction: the feature improves prediction of how behaviour changes under controlled prompt or activation edits.

The higher rungs require more work because the scientific claim becomes stronger.

65. Failure signatures for superposition analysis

Failure 1: calling every polysemantic neuron superposition

A neuron has several activation themes, so the researcher declares superposition without showing a better feature decomposition.

Repair: compare the neuron basis with learned directions, probes and causal tests.

Failure 2: treating an SAE feature as ground truth

The latent is interpretable, so its human label is assumed to be the unique concept stored by the model.

Repair: test feature splitting, alternative dictionaries and causal specificity.

Failure 3: ignoring reconstruction residual

The dictionary explains 90% of activation variance and the unexplained 10% is silently treated as irrelevant.

Repair: measure whether residual directions carry task-relevant behaviour.

Failure 4: selecting only beautiful features

The paper highlights a dozen crisp features from a dictionary containing millions.

Repair: sample features randomly, report interpretability distributions and dead/unclear feature rates.

Failure 5: forgetting feature density

A very rare feature appears monosemantic because only a handful of examples exist.

Repair: report activation frequency and confidence intervals around interpretation tests.

Failure 6: measuring only top activations

The strongest examples fit the label, but medium activations do not.

Repair: inspect activation across the full range and use held-out labelled examples.

Failure 7: oversteering

A huge activation intervention produces the labelled behaviour, but the model is far outside its normal state distribution.

Repair: use dose–response curves and compare intervention magnitudes with natural activations.

Failure 8: treating feature count as capacity without quality

A 10-million-feature dictionary is assumed better than a 1-million-feature dictionary.

Repair: compare reconstruction, sparsity, interpretability, stability and causal usefulness at matched budgets.

Failure 9: assuming cross-layer identity

A feature labelled “deception” in layer 10 is assumed identical to a “deception” feature in layer 20.

Repair: test representational correspondence and downstream role separately at each layer.

Failure 10: using the atlas as an explanation of behaviour

A relevant feature activates during a behaviour, so the behaviour is declared explained.

Repair: identify the causal path and test counterfactual predictions.

66. Worked example: four sparse features in a two-dimensional code

Consider four binary features A, B, C and D.

Each is active independently with probability 0.01.

We have only two activation dimensions.

A one-feature-per-axis system can represent only two features cleanly.

A superposed system assigns four directions around the plane.

For most inputs, zero or one feature is active.

When only A is active, the activation points near d_A.

When only B is active, it points near d_B.

And so on.

The rare collision A+B produces d_A+d_B. The downstream system must tolerate or resolve this mixed state.

Because simultaneous activation occurs with probability roughly 0.0001 for any independent pair, the expected interference cost can be much smaller than the value gained by storing all four features.

Now change the data so A and B always occur together.

The interference cost becomes frequent.

The model may allocate cleaner directions, merge A and B into a composite feature, or choose another representation.

The example captures the essence: superposition is an optimization response to feature sparsity, co-occurrence, importance and available dimension.

67. Worked example: why one neuron can look like three concepts

Suppose a three-dimensional activation contains six sparse feature directions.

Feature F₁ points mostly along neuron 1 and slightly along neuron 2.

Feature F₂ also has a positive neuron-1 component but differs strongly on neurons 2 and 3.

Feature F₃ has a smaller neuron-1 component.

If F₁ represents French, F₂ represents legal citations and F₃ represents URLs, neuron 1 can activate for all three.

Looking only at neuron 1 produces a polysemantic story.

Looking at the full activation vector reveals three distinguishable directions.

A sparse dictionary can learn those directions and make each feature more monosemantic than the raw neuron.

This is the central interpretability promise of decomposing superposition.

68. What superposition does not prove

• It does not prove every polysemantic neuron arises from sparse feature packing.
• It does not prove every meaningful internal variable is a linear direction.
• It does not prove sparse autoencoders recover the unique ground-truth feature set.
• It does not prove a human-readable feature is causally used by the model.
• It does not prove feature labels generalize outside the inspected data.
• It does not imply biological brains use the identical mechanism.
• It does not mean more features are always better; interference can reduce performance.
• It does not imply a model with wider layers is automatically more interpretable.

These limits make the theory stronger, not weaker. They separate a useful mechanistic hypothesis from mythology.

69. Boundary with Sparse Autoencoders

This article owns the why of feature crowding: why sparse features may be represented in superposition, what geometry and interference mean, and why raw neurons become polysemantic.

The next queued owner, Sparse Autoencoders for Neural Representation, owns the how to decompose job: encoder/decoder architecture, overcomplete dictionaries, sparsity objectives, TopK and L1 variants, dead latents, reconstruction fidelity, feature evaluation, steering, scaling and failure modes.

Keeping these owners separate prevents the method from swallowing the theory. A reader can understand superposition without committing to one interpretability tool, and can study sparse autoencoders without treating superposition as already proven in every model.

70. Mathematical appendix: capacity, interference and sparse recovery

Superposition becomes easier to reason about when three quantities are separated: the number of candidate features F, the activation dimension d, and the expected number k of simultaneously active features.

If F ≤ d, the model can in principle assign an orthogonal direction to every feature, though architecture and optimization need not choose that solution.

If F > d, all feature directions cannot be mutually orthogonal. Some overlap is unavoidable if all F are represented linearly in the same d-dimensional space.

The cost of overlap depends strongly on k. When k is small relative to F, each input samples only a tiny subset of possible collisions. When k grows, many interfering directions contribute at once and sparse packing becomes less attractive.

In compressed sensing, sparse vectors can sometimes be recovered from fewer measurements than ambient coordinates under appropriate assumptions on the measurement matrix and sparsity. Neural-network superposition is not identical to compressed sensing, but the analogy is useful: sparsity lets a lower-dimensional observation preserve more candidate variables than dense worst-case reasoning would allow.

The analogy also exposes the caveat. Recovery guarantees require assumptions such as incoherence or restricted isometry that real learned neural representations do not automatically satisfy. One should not import a theorem from compressed sensing and declare a transformer solved.

71. The Gram matrix reveals which feature directions interfere

Let D be a matrix whose columns are normalized feature directions d₁,…,d_F.

The Gram matrix is:

G = DᵀD.

Gᵢᵢ = 1 for normalized directions.

Off-diagonal entries Gᵢⱼ measure cosine overlap between feature directions.

An orthogonal feature code has off-diagonal zeros.

A superposed code has non-zero overlaps.

The sign matters. Positive overlap makes features reinforce one another under projection. Negative overlap can create cancellation or allow antipodal packing when the feature activation constraints make that useful.

The Gram matrix therefore gives a compact picture of representational crowding. It does not tell us which feature pairs co-occur or how nonlinear layers respond, so it is only one part of the interference analysis.

72. Expected interference depends on geometry and co-activation together

Suppose the activation is x = Σᵢ aᵢdᵢ.

Projecting onto dⱼ produces the target coefficient aⱼ plus interference terms proportional to aᵢ(dⱼ·dᵢ).

The expected squared interference depends on:

• feature activation magnitudes,
• co-activation probabilities,
• pairwise directional overlaps,
• higher-order dependencies among active features.

This formalizes the “bad roommates” idea. A large cosine overlap is harmless if the two features never appear together. A modest overlap can be expensive if they co-occur constantly and feed a sensitive downstream decision.

A principled superposition analysis should therefore estimate both feature geometry and empirical joint activation, not just one.

73. Dictionary non-uniqueness is not a nuisance detail; it shapes the ontology of features

An overcomplete dictionary can represent the same activation in several ways unless the objective and data distribution select a stable sparse solution.

This means two well-trained sparse autoencoders can achieve similar reconstruction and sparsity while learning somewhat different feature sets.

Some features may match cleanly across runs.

Some may rotate inside a subspace.

Some may split or merge.

Run-to-run stability is therefore a scientific observable.

A feature that appears only under one random seed deserves weaker ontological language than a feature that recurs across many independent dictionaries, activates on the same concept, and has the same causal role.

74. “How many features does a model have?” may not have one scalar answer

The question sounds like asking how many neurons the model has.

Feature count is less definite.

At coarse resolution, one dictionary might describe 100,000 broad factors.

At finer resolution, the same activation space might support millions of narrower features.

Curved or conditional features can require multiple linear dictionary elements.

Composite features can be counted as wholes or as combinations of parts.

A more useful scientific question is often: at this layer and resolution, how many sparse learned factors are needed to reconstruct and explain the behaviours we care about?

75. Feature importance can be measured several ways, and they need not agree

A feature can be frequent but causally weak.

Another can be rare but decisive.

Importance metrics include activation frequency, activation magnitude, reconstruction contribution, downstream logit effect, behavioural ablation effect, loss change under intervention and relevance to a declared task.

These measures answer different questions.

A frequent punctuation feature can explain lots of activation variance and little semantic behaviour.

A rare safety-relevant feature can explain little global variance and matter enormously when it activates.

Superposition theory predicts capacity allocation should reflect the training loss, not human interpretability priorities. A safety auditor may therefore care most about features the model allocated little clean capacity to.

76. Rare features are both the best case for superposition and the hardest case for science

Rare features create low interference, making them attractive candidates for dense packing.

They also provide few examples for interpretation.

A latent that fires twenty times in ten billion tokens can look perfectly coherent by chance.

Its causal tests can be underpowered.

Its feature density estimate can be noisy.

Rare-feature science therefore needs targeted data generation, synthetic probes, adversarial examples and careful uncertainty reporting.

The most interesting safety features may be exactly the ones ordinary corpus browsing almost never sees.

77. Failure to find a feature is not strong evidence the model lacks the concept

An SAE can fail to recover a concept for several reasons.

• The concept may be represented nonlinearly.
• The dictionary may be too small.
• The feature may be too rare in the activation dataset.
• The sparsity penalty may merge it with another feature.
• The chosen layer may not represent the concept cleanly.
• The representation may use a circuit state rather than one direction.
• The feature may live primarily in another activation site.

Interpretability tools therefore have recall as well as precision.

A dictionary that finds many beautiful features can still miss important ones.

78. Feature coverage needs behavioural tests, not only reconstruction percentage

High reconstruction fidelity means the dictionary approximates activations well on average.

It does not guarantee coverage of every behaviourally important feature.

A rare direction can contribute negligibly to mean squared reconstruction error and be essential for a rare behaviour.

Coverage evaluation should therefore include challenge sets targeting known capabilities, rare behaviours and safety-relevant states.

For each behaviour, ask whether candidate features predict and mediate the behaviour sufficiently to support the intended interpretability claim.

Average reconstruction is a necessary engineering metric, not a complete scientific coverage certificate.

79. Superposition analysis is moving beyond language models

Sparse feature decomposition has begun to appear in scientific foundation models and recommender systems, not only language models.

Recent 2026 work applied sparse-autoencoder interpretability to a neutrino foundation model and identified candidate latent physical concepts, then used causal interventions to test how downstream heads used those features.

Other work explores SAE-derived interpretable intents for recommendation.

These applications matter because they test whether feature decomposition generalizes across data modalities and architectures.

They also raise stronger domain-validation requirements. A feature labelled with a physical concept should be tested against nuisance variables, held-out scientific conditions and domain theory rather than only language-model descriptions.

80. Neural-network superposition should not be silently imported into biological neuroscience

Biological neural populations also use distributed, sparse and mixed representations.

There are mathematical analogies between population coding and feature packing.

But biological neurons differ from transformer activations in architecture, dynamics, noise, connectivity, plasticity, metabolic constraints and measurement.

The specific mechanistic-interpretability hypothesis developed in transformer toy models should therefore remain an AI representation claim unless evidence establishes an analogous biological mechanism.

The Neural Population Geometry owner remains the correct route for biological population coding.

81. Advanced FAQ: superposition in neural representations

Is superposition just distributed representation?

No. Distributed representation means one feature uses several units. Superposition means several candidate features share a space with fewer independent dimensions than clean orthogonal representation would require.

Is every polysemantic neuron evidence of superposition?

No. Polysemanticity is consistent with superposition but can arise from other forms of contextual reuse or nonlinear computation. Stronger evidence requires a feature decomposition and mechanistic validation.

Why does sparsity help?

Because features that are rarely active together interfere rarely. The model can reuse dimensions across them and pay the collision cost only on uncommon co-activations.

Can a model have more features than neurons?

Yes in the sense used by superposition and dictionary-learning work: candidate feature directions can outnumber architectural neurons or activation dimensions when features share those dimensions sparsely.

Does a sparse autoencoder prove superposition?

No. An SAE can provide evidence that a sparse overcomplete decomposition explains activations better than raw neurons for particular jobs. The inference to superposition requires additional theoretical and empirical support.

What is a monosemantic feature?

It is a learned unit or direction whose activation has a relatively coherent interpretation across contexts. Monosemanticity is graded and depends on feature resolution and evaluation.

Why can one feature split into several?

A larger dictionary has enough capacity to separate distinctions that a smaller dictionary merged. Splitting can also approximate curved or context-dependent structure with several linear directions.

Does steering a feature prove the model naturally uses it?

No. Steering shows causal leverage under intervention. Natural-use claims need evidence about ordinary activations, downstream reading and necessity or counterfactual prediction.

Can superposition improve model performance?

It can increase representational capacity when feature sparsity makes interference affordable. If features are dense or collision costs are high, superposition can hurt and cleaner representation may be preferable.

Is there one true number of features in a model?

Probably not in a simple resolution-independent sense. Feature count depends on layer, dictionary resolution, linearity assumptions, context and the scientific job used to define meaningful factors.

What would falsify a superposition story?

Evidence that meaningful variables align cleanly with independent architectural dimensions, that overcomplete sparse dictionaries add no explanatory value, that apparent polysemanticity disappears under better controls, or that proposed feature decompositions fail causal and cross-run validation would weaken a superposition account for that representation.

82. Research references and reading route

[1] Elhage, N. and colleagues (2022). Toy Models of Superposition. A foundational mechanistic exploration of how sparse features can be packed into lower-dimensional neural representations and how feature importance and sparsity shape geometry.

[2] Bricken, T. and colleagues (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Demonstrates sparse-autoencoder feature decomposition in a small transformer and documents interpretability, feature splitting, universality and causal interventions.

[3] Templeton, A. and colleagues (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Extends sparse-autoencoder feature extraction to a much larger language model.

[4] Wu, J., Gao, L., Dupré la Tour, T. & Tillman, H. (2024). Extracting Concepts from GPT-4. Reports large-scale sparse-autoencoder training and explicitly discusses remaining limits in coverage and validation.

[5] Anthropic (2026). Would This Change Your Answer? Evaluating Explanations of LLM Behavior in the Wild with Counterfactual Experiments. A useful current caution: activation-reading tools including SAEs did not improve one counterfactual behaviour-prediction benchmark over a transcript-only baseline.

[6] Gupte, S., Zhang, X. & Khalili, M. M. (2026). When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs. Studies how post-hoc model pruning can alter SAE behaviour and emphasizes activation geometry.

[7] Bonnet-Guerrini, R. and colleagues (2026). Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders. A recent example extending SAE-style interpretability into scientific foundation models with causal validation.

83. The library route from representation to decomposition

• Representation — what stands in for something else.
• Feature — a variable or property used to distinguish states.
• Dimension — an axis along which a representation can vary.
• Superposition — the broader idea of additive overlap in signals and state descriptions.
• Superposition in Neural Representations — why sparse learned features may share activation dimensions in neural networks.
• Sparse Autoencoders for Neural Representation — the next owner, explaining one major method for decomposing dense activations into sparse candidate features.
• Representation Similarity Analysis — a later owner, explaining how representational structures can be compared without requiring one-to-one coordinate correspondence.

World Return: the neuron was never promised to be the concept

The seductive mistake is to look at a neural network and assume the architecture’s visible coordinates are the mind of the model.

They are not.

They are the coordinates optimization was given.

The useful variables learned inside that coordinate system can be rotated, distributed, composed and packed together.

Superposition is one explanation for why this packing can be economically rational: the model lives in a world with far more potentially useful sparse features than any one layer has clean independent axes.

The bargain is capacity in exchange for interference.

Mechanistic interpretability tries to reverse that bargain for the human reader. It searches for feature directions, dictionaries and circuits that recover structure hidden by the shared coordinates.

The hardest part is no longer finding a beautiful feature. It is establishing that the feature is stable, specific, causally relevant and sufficient to improve our prediction of what the model will do.

The neuron was never promised to be the concept. Superposition is the theory that tells us why—and the measurement problem that follows.

84. Longform audit: why this owner needs depth

A short glossary entry can define superposition as “more features than dimensions.” That definition is not enough to use the concept safely.

The reader must distinguish features from neurons, superposition from distributed representation, sparsity from sparse neurons, capacity from feature count, polysemanticity from proof, dictionary features from ontological ground truth, reconstruction from behaviour, steering from natural use, and causal leverage from explanatory completeness.

The practical reader also needs collision tests, feature-density diagnostics, reconstruction residual analysis, feature-splitting checks, cross-run stability, cross-model matching, versioning after fine-tuning or pruning, safety-monitoring limitations and evidence ladders for claimed mechanisms.

That functional coverage is why the article belongs at longform scale. The word count is not the achievement. The achievement is one canonical page that carries the entire reader job without forcing the concept into a chain of thin, cannibalising definitions.