tl;dr: large language models are opaque by nature, a black box. sparse autoencoders are a promising technique for decomposing their inscrutable neural activations into something closer to concepts a human can actually read, making them translucent if not fully transparent. this is the first in a series on how they work and why they matter.
understanding neural networks has been a struggle for researchers since they started getting bigger. after models like alexnet, performance improved drastically while understanding what these networks were actually learning got much harder. with llms, understanding the internals is tougher still. but it matters: these systems are being deployed in healthcare, legal settings, and education, contexts where knowing why a system produced a particular output isn't just interesting, it's practically essential.
sparse autoencoders, or saes, are one approach to what's called mechanistic interpretability: the project of reverse-engineering a neural network to understand not just what it does, but how it does it.
i. the problem: polysemanticity
before we can understand what saes do, we need the problem they're solving. in a language model, information is represented as vectors that encode the "meaning" of words, phrases, and concepts. these vectors live in a latent space, and get transformed through layer after layer of the network.
naively, we might expect individual neurons to correspond to individual concepts: one neuron for "cats", another for "democracy". neat, interpretable, human-readable. reality doesn't cooperate. researchers have found that individual neurons in large language models are polysemantic, they respond to multiple, seemingly unrelated concepts at once. a single neuron might fire strongly for academic citations, english dialogue, http requests, and korean text, not because those things are conceptually related, but because the network has learned to compress an enormous amount of information into a limited number of neurons.
this is called superposition: a model needs to represent far more concepts than it has neurons, so it encodes multiple concepts as overlapping patterns across the same neurons, using different combinations and weightings to tell them apart. extraordinarily efficient computationally, but inconvenient if you're trying to figure out what the model is actually doing.
ii. untangling the mess
this is where sparse autoencoders come in. the core idea is simple: take the tangled, superimposed representations inside a model and untangle them into something interpretable, decomposing the mess into clean, separate features that each correspond to a single concept.
an autoencoder, generally, is a network that learns to compress data and reconstruct it: feed it an input, compress it through a bottleneck (the "encoding"), then try to rebuild the original from that compressed form. if it can reconstruct successfully, the bottleneck must contain the essential information, just in a more compact shape.
sparse autoencoders flip this: instead of compressing into a smaller representation, they expand into a much larger one. the constraint isn't size, it's sparsity, for any given input, only a small number of features in that expanded representation should be active.
say a model has hidden representations of dimension 4,096. an sae trained on it might learn a representation of dimension 131,072, thirty-two times larger, but for any particular input, only a few dozen of those 131,072 features actually turn on.
sparsity helps because it forces disentanglement. when a model has to represent each input using only a handful of features from a big dictionary, those features tend to specialize. instead of polysemantic neurons responding to everything, you get monosemantic features that each correspond to one specific, interpretable concept.
given an input representation z from the language model, the sae encodes it as:
f = σ(W_enc·z + b_enc)
W_enc maps from the model's dimension to the expanded one; σ is an activation function that enforces sparsity
the decoder then reconstructs the original representation from those features:
ẑ = W_dec·f + b_dec
training balances two goals in tension: reconstruct the original activation as accurately as possible, and keep the encoding as sparse as possible. more active features usually means better reconstruction but worse interpretability, finding the right balance is one of the central challenges in sae research.
iii. what the features look like
when researchers at anthropic trained saes on claude 3 sonnet, one of the more capable models around at the time, they found features corresponding to remarkably specific concepts: not vague, abstract patterns, but things like "sycophantic praise", "discussions of scientific methodology", "references to the golden gate bridge", and "mentions of the specific year 2025".
you can verify a feature corresponds to a concept by checking which inputs activate it most strongly, a technique called maxact, still one of the main ways researchers interpret what an sae feature means. there's also an output-based approach: because language models share the same representational space across layers, you can project sae features onto the model's output vocabulary and see which words they most strongly predict. a feature that strongly predicts "food", "restaurant", "cooking", and "ingredient" is probably encoding something about food.
the practical implications matter. if we can identify which features correspond to which concepts, we can potentially steer a model's behaviour by artificially activating or suppressing them, turn down the sycophancy feature, or amplify the safety-relevant ones so the model refuses harmful requests more reliably. this isn't hypothetical: researchers have already shown sae-derived steering vectors can meaningfully shift model outputs for concepts like toxicity, emotional tone, and refusal behaviour.
iv. the limitations
reconstruction is never perfect. replacing a model's original activations with sae-reconstructed ones degrades performance, recent research suggests this can be equivalent to training the original model with only 10% of its compute.
the concept dictionary is never complete, either. saes learn features from the data they're trained on, so rare concepts may be poorly represented or missing outright. the computational cost adds up too: saes need training separately for each layer, often with feature dimensions many times larger than the original, which for a model with dozens of layers and billions of parameters is not a weekend project.
and perhaps most fundamentally, the theoretical foundations are still shaky. we assume concepts are represented linearly. we assume superposition is the primary obstacle to interpretability. these assumptions hold up empirically, at least partially, but we don't yet have the mathematical framework to prove they're correct, or to know exactly when they fail.
v. where this is going
the question isn't whether saes are perfect today, it's whether they're headed toward genuine usefulness, and the signs are good. in the past two years alone, saes have scaled from small models to production-grade ones. architectural improvements like gated saes, topk saes, and jumprelu saes better balance reconstruction fidelity against sparsity. and public resources like neuronpedia now let anyone explore the features an sae has learned on popular models.
next up: the different sae architectures and their trade-offs, how to evaluate and explain features, and how all of this feeds into model steering and safety research.
for now: sparse autoencoders are a technique for decomposing the tangled, polysemantic representations inside language models into cleaner, more interpretable features. and as ai systems get more consequential, understanding what they're actually doing seems like it matters.