Feature disentanglement is the goal of turning a model’s tangled internal activations into a set of clean, individually meaningful features — ideally one concept per axis, so that reading or editing a feature does not disturb its neighbours. It is the mathematical heart of mechanistic interpretability: if a hidden vector really were a simple sum of independent concept directions, we could read the model’s mind with a dot product. Reality is messier. Models pack more features than they have dimensions, so concepts share directions and interfere; and even when a clean decomposition exists, it is not unique — a rotation can scramble the axes without changing the model at all. This piece works through the linear representation hypothesis, the interference math that superposition forces on us, why disentanglement is provably ill-posed without extra structure, how sparsity rescues it, and the metrics we use to say whether we succeeded.
What disentanglement actually means
A representation is disentangled when the model’s generative factors — the underlying concepts that vary in the data — map one-to-one onto separate, recoverable components of the activation. Change one factor in the world and exactly one component of the vector moves; the rest hold still. An entangled representation smears each factor across many components and mixes many factors into each, so no single number means any single thing.
The classic informal test is the ‘knob’ test: if you could grab one coordinate and turn it, would a human-interpretable property change on its own? In a language model, one direction for ‘this text is in French,’ one for ‘the subject is plural.’ Disentanglement is the difference between a representation you can describe and one you can only use — because a disentangled feature is a unit you can name, monitor, ablate, and reason about causally, and an entangled one is not.
The linear representation hypothesis: features as directions
The dominant working assumption is the linear representation hypothesis: a concept is encoded as a direction in activation space, and the presence of several concepts is encoded as a weighted sum of their directions. Formally, for a hidden vector x in R^d and a set of feature directions d_1, …, d_n,
x ≈ Σ_i a_i · d_i a_i ≥ 0 (activation of feature i)
d_i ∈ R^d, ||d_i|| = 1 (unit feature direction)This is a strong claim, and a surprisingly well-supported one: linear probes recover concepts, steering vectors add and subtract meaning, and analogies behave like vector arithmetic. The pay-off is that reading a feature becomes a projection onto its direction, and disentanglement becomes a concrete question about the geometry of the {d_i}. If the directions are orthonormal, the features are perfectly disentangled. Every difficulty below is a way that assumption fails.
Reading a feature: the interference term
Suppose you want to read feature i from x with a readout direction w_i. The natural choice is w_i = d_i. Then
d_i · x = d_i · (Σ_j a_j d_j)
= a_i (d_i·d_i) + Σ_{j≠i} a_j (d_i·d_j)
= a_i + Σ_{j≠i} a_j (d_i·d_j)
^ signal ^ interferenceThe second sum is the interference term, the crux of the whole subject. If the directions are mutually orthogonal, every d_i·d_j = 0 and the readout is exact: d_i·x = a_i — a fully disentangled representation. But orthogonality allows at most d features in d dimensions. The instant you have more features than dimensions — which real models always do — some d_i·d_j ≠ 0, the interference term is nonzero, and reading one feature picks up ghosts of the others. Entanglement is exactly this cross-talk.
Superposition: why the model entangles on purpose
Superposition is the reason interference is unavoidable. A model has vastly more concepts it might represent than it has neurons, so it stores n > d features in d dimensions by giving them non-orthogonal directions. This is not a bug; it is an efficient code. High-dimensional geometry makes it nearly free: by the Johnson–Lindenstrauss lemma you can fit exponentially many almost-orthogonal unit vectors in R^d, with pairwise dot products of order 1/√d.
The trick that makes superposition tolerable is sparsity. If only a handful of features are active on any input, the interference sum Σ_{j≠i} a_j (d_i·d_j) has few nonzero terms, each small, so noise stays below the signal a_i. The model is doing lossy compression: it bets its features rarely co-occur and packs them accordingly. Superposition is thus the mechanism that creates entanglement, and disentanglement is the effort to undo it — to recover the individual d_i from activations where they were folded together.
Disentanglement as the inverse of superposition
Cast this way, disentangling a layer is an unmixing problem. We observe x = D a, where D is the d × n matrix whose columns are the feature directions and a is a sparse, nonnegative vector of true activations. We see only x; we want to recover both D and a. This is dictionary learning / sparse coding, the formal problem a sparse autoencoder is trained to solve (see the sibling articles).
The important point is that disentanglement is overcomplete recovery: we deliberately look for more features n than dimensions d, because that is how the model stored them. An orthonormal change of basis (PCA, a rotation) can never do this — it is stuck with d orthogonal directions and cannot separate superposed features sharing a subspace. You need a method that exploits sparsity, not just variance — which raises the question of whether the answer is even unique.
The rotation problem: why the naive version is ill-posed
Here is the subtlety that defeats naive approaches. Suppose the true model is linear, x = D a, and the true factors a happen to be independent and Gaussian. Take any orthogonal rotation R (so R^T R = I) and write
x = D a = (D R^T)(R a) = D′ a′The rotated a′ = R a are still independent, still Gaussian, still unit-variance — statistically indistinguishable from the originals — and D′ = D R^T reconstructs x exactly as well. Nothing in the data prefers the true axes over the rotated. This is the rotation problem (the non-identifiability of Gaussian factor models and linear ICA under Gaussian sources). Locatello et al. (2019) sharpened it into a theorem: unsupervised disentanglement is impossible without inductive biases or supervision. Any purely reconstruction-based objective has a whole orbit of equally good, differently-rotated solutions, and only one of them lines up with the concepts a human would name.
What breaks the symmetry: sparsity and non-Gaussianity
The rotation degeneracy is not the end of the story, because it depends on a special assumption — Gaussian, rotation-symmetric factors. Real features are not like that. They are sparse and heavy-tailed: mostly off, occasionally strongly on. Sparsity is not rotation-invariant — a rotation of a sparse vector is generically dense — so among all the rotated solutions, the sparse one is special and recoverable.
This is exactly the classical ICA insight (recover sources by maximizing non-Gaussianity) and the sparse-coding insight (an L1/L0 penalty on a selects the sparse basis). The inductive bias a sparse autoencoder injects — reconstruct x using as few active features as possible — is precisely what pins down which rotation is right. So the recipe is: assume the linear representation hypothesis, then break the rotation symmetry with a sparsity prior. Without it, you chase an answer that mathematically does not exist.
A worked example: three features in two dimensions
Concreteness helps. Put n = 3 features into d = 2 dimensions as unit vectors at 0°, 120°, 240° — the tightest symmetric packing. Every pair has d_i·d_j = cos(120°) = −0.5.
d_1 = (1, 0) d_2 = (−0.5, 0.866) d_3 = (−0.5, −0.866)
one feature on: a = (1, 0, 0) → x = (1, 0)
read f_1: d_1·x = 1.0 (exact, no interference)
two features on: a = (1, 0.8, 0) → x = (0.6, 0.693)
read f_1: d_1·x = 1 + 0.8·(−0.5) = 0.6 (0.4 of ghost f_2)When features fire one at a time — perfect sparsity — the readout is exact even though the directions are far from orthogonal: superposition is free. The moment two co-occur, the −0.5 overlap leaks a 0.4 error into the reading of f_1. That single number, the interference, is disentanglement quality made visible: it scales with the dot-product between directions and with how often features co-activate.
Metrics: how do you know you disentangled anything?
Because the answer is non-unique, you cannot judge disentanglement by reconstruction loss alone — you need metrics that reward one-concept-per-axis. When ground-truth factors are known, MIG (Mutual Information Gap) measures, per factor, the gap in mutual information between its top-two latents — a big gap means one latent owns the factor — and the DCI triple (Disentanglement, Completeness, Informativeness) and SAP score the same idea differently.
In mechanistic interpretability we rarely have ground-truth factors, so proxies stand in: the sparsity of the code (mean L0, active features per token), the reconstruction fidelity (variance explained, or loss recovered when the reconstruction is spliced back in), the interference / maximum cosine similarity between learned feature directions, and interpretability scores for whether a feature fires on a coherent, describable set of inputs. Good disentanglement is a Pareto frontier: high fidelity, low L0, near-orthogonal, nameable features.
Why clean features matter, and the CPU-SLM angle
Disentanglement is not just aesthetics. A disentangled feature is a lever: you can steer by adding α d_i to activations, ablate by projecting it out, and monitor by watching a_i — and because the feature is isolated, these edits do not spray side effects across unrelated behaviour. Entangled directions make every intervention blunt.
For small models on CPU the stakes are sharper. A tighter dimension budget forces more superposition, so an SLM’s features are more crowded and harder to separate. But that same budget is what makes disentanglement valuable: a lightweight linear probe or steering vector runs at near-zero inference cost, catching a spurious feature and correcting behaviour without retraining or a bigger model. Interpretability is one of the few ‘free’ levers left when you cannot simply add parameters.
Pitfalls and open problems
A few traps recur. Assuming linearity everywhere: the feature-as-direction picture is an approximation; some concepts live on nonlinear manifolds or in multi-dimensional subspaces (a ‘day of the week’ circle, not a single axis), and forcing them onto one direction disentangles nothing. Confusing PCA with disentanglement: principal components are the high-variance orthogonal directions — a different objective that merges distinct rarely-varying concepts and cannot go overcomplete.
Reading fidelity as understanding: a decoder can reconstruct x beautifully with features that are dense, redundant, or uninterpretable — low loss is necessary, not sufficient. And the deepest open problem remains identifiability: we break the rotation symmetry with sparsity because it works, but proving that the recovered features are the model’s true computational units, rather than a convenient sparse basis, is still unsettled.