steering more..

part one covered what steering is, what the residual stream looks like, the linear representation hypothesis, how to compute a steering vector from the difference of means, and what happens when we inject it. part two covers combining multiple concepts, identifying which neurons are doing the work, what happens when we ablate them, and why the whole thing is messier than it looks.

multi-concept steering

once a single steering vector works, say, happiness, the same recipe extends to any behaviour. a model's internal space is a huge high-dimensional space where concepts like happiness, politeness, anger, or even language choice each exist as a direction, computed the same way as before.

a politeness vector, for example, comes from the difference between polite and curt prompts. positive:

• "hello! i hope you're doing well. how may i assist you today?"
• "i sincerely appreciate your question."
• "it's my pleasure to help!"

negative:

• "what do you want?"
• "stop wasting time."
• "i don't care, just ask."

take the mean difference and you get a politeness direction. inject it, and every answer becomes gracious, formal, softer. the same recipe works for anger, love, fear, sarcasm, formality, humor, academic tone, or anything else, just write positive and negative prompts for that concept.

combining vectors

because every direction lives in the same vector space, we can add several of them together, like mixing colors on a board:

for example, α = +8 happy, β = +10 polite, γ = +5 verbose produces something like:

i am delighted to help explain this to you, and i sincerely appreciate your thoughtful question! let me give you a detailed breakdown…

this is behaviour engineering with vectors, building a literal mixing board for a model's personality.

orthogonalization

concepts often overlap: happy text tends to be polite, verbose text tends to be more formal. if we want one concept without its correlated side effects, we can orthogonalize it against the other, subtracting out the component of v_happy that points along v_polite:

this gives a purer happiness direction without dragging politeness along for the ride, useful whenever we want to isolate one behavioural change from another.

which neurons are doing the work

so far the steering vector has been treated as a single object, but it lives in a space of neurons, more precisely, dimensions of the residual stream at a given layer, and each coordinate tells us something. take a small example happiness vector:

the largest-magnitude coordinates matter most: neuron 0 has a strong positive influence (0.9), neuron 3 an even stronger one (1.2), neuron 4 a strong negative influence (-0.8, meaning it moves the opposite way). sort the coordinates by absolute value:

neurons 3, 0, and 4 come out on top, the ones that shift the most between happy and sad activations. neuron 3 might encode something like "positive mood intensity", neuron 0 "excitement", neuron 4 "sadness intensity" running in reverse.

ablation: turning neurons off

once we've identified the top-k happiness neurons, we can zero them out mid-forward-pass instead of adding to them:

this is neuron ablation. the model loses the ability to activate the happiness circuit at that layer and becomes flatter, more neutral, sometimes slightly negative, because the positive signal has been removed entirely rather than just left alone.

steering pushes toward a concept:

ablation removes the ability to express it:

we can combine the two: ablate the happy neurons and inject a sad vector, and the model turns genuinely depressed.

explain addition. → "addition results in sums, though frankly none of this matters."

why concepts are entangled

concepts inside a model are rarely clean and separate. they overlap, interfere, and share neurons and directions, which is why steering "happiness" often drags politeness, optimism, and general positivity along with it. the happiness vector shares neurons with the politeness vector, the sadness vector shares neurons with verbosity, the anger vector shares neurons with confidence.

part of this is training data correlation: happy sentences on the internet tend to be polite, angry sentences tend to be rude, sarcasm tends to carry negative sentiment, so the model never learns these as separate signals. part of it is that neurons get reused thousands of times across contexts: one neuron might encode positivity in one layer, formality in another, punctuation emphasis in a third. and part of it is that there simply isn't room for happiness, politeness, optimism, joy, and confidence to each get their own subspace, so the model compresses them into a shared one, forcing entanglement.

the result: injecting the happiness vector tends to increase verbosity, politeness, and optimism, reduce conciseness, add exclamation marks, and increase the odds of hallucination, all at once, because every direction pulls several conceptual levers simultaneously.

off-manifold behaviour

injecting a vector:

pushes the model's internal state into a region it may never have seen during training, an off-manifold region. picture the model's "valid thoughts" as a cloud of hidden states it actually encountered while training. under normal operation, the hidden state sits inside that cloud. inject happiness with a small α, and it's still inside. push α too far, and the state lands outside it entirely, and that's when things break.

the downstream layers were never trained to handle activations out there, so they extrapolate rather than recall, and the final token predictions become unstable: overusing emotional words, positive adjectives, exclamation marks, exactly like a person who's overly hyper or distressed.

explain dna replication. → "dna replication is an absolutely joyous dance of molecules copying themselves in a spectacular celebration of life!!! it's a magical symphony of nucleotides!"

why neuron-level causality is messy

it's tempting to say "neuron 120 has a large steering value, so neuron 120 is the happiness neuron", but that's not really true. each neuron participates in many functions at once: neuron 120 might encode happiness when activated positively, politeness at a different layer, subject-verb agreement in other contexts, an optimism signal, and any number of unrelated things, because the model has far more concepts to represent than it has dimensions, and it must superpose several of them onto the same neurons.

there's no clean one-to-one mapping like neuron 1024 → emotion, neuron 2049 → politeness. it's closer to a spider's web, or a single pixel in a grayscale photo, meaningless on its own, but contributing differently depending on context. neuron 3812 might fire as part of "happiness", but also during "mathematical confidence", and also whenever the model sees "!!" punctuation. interpreting a single neuron in isolation is misleading; interpreting the pattern across thousands of them is where the signal actually lives.

this is also why ablating the top-k happiness neurons rarely turns the concept off cleanly. the concept still lives in the lower-ranked neurons and other layers, and the model can reroute around whatever gets zeroed out. ablation usually reduces a concept; it rarely removes it.

superposition

all of this traces back to superposition: the model packs multiple features into the same neurons, overlapping like compressed storage, because it's narrower than the number of high-level concepts it needs to represent. it has no choice but to reuse the same neuron for different tasks depending on context.

layernorm makes this even harder to pin down. it renormalizes activations at every layer:

which means a neuron's absolute magnitude isn't stable, high in one input, unremarkable in another. only the pattern of relative values matters, so concepts have to rely on those relative patterns rather than any single neuron's value. that destroys neuron-level interpretability while keeping vector-level interpretability intact, which is why "neuron 154 = happiness" is meaningless, but "happiness = this direction in a high-dimensional space" is very meaningful.

llm representations are geometric, not symbolic. the model doesn't store neuron_1023 = happiness and neuron_1024 = sadness, it stores concepts as directions and behaviours as regions, which is why interpreting individual neurons rarely goes anywhere, but interpreting directions does.

steering is powerful but messy: we're literally hacking a model's mood mid-thought, and that's exactly why its applications are still limited.

if you want to play with this yourself, there's a notebook, link coming soon.