← Back to Blog

Oct 23, 2026 · 4 min read

Abliteration: What Is It?

LLMsInterpretability

The other day I came across a term that I thought sounded cool, ABLITERATION. So what is it? Chat models are trained to refuse certain requests. The process of abliteration strips that refusal out by editing the model's weights directly, without retraining it. The name is a blend of ablation (removing a part of a model to see what it does) and obliterate. It started as the tag FailSpy put on their uncensored models on Hugging Face, and it stuck. The method is more precise than the name suggests, and it's a neat way to see how refusal is represented inside a model.

How it works

It rests on one finding from Arditi et al. (2024), who showed that across 13 open chat models, refusal is largely mediated by a single direction in the model's activation space, the "refusal direction". Erase it from the activations and the model stops refusing harmful requests. Add it to the activations for a harmless prompt and the model refuses that too.

To find the direction, you run the model on a set of harmful prompts and a set of harmless ones (the paper used 128 of each) and take the difference between their mean activations:

r = mean activation on harmful prompts − mean activation on harmless prompts

The difference gives one candidate direction per layer, and per token position after the instruction. Each candidate is tested by erasing it and checking whether the model still refuses. The paper keeps the one that stops refusals best without changing much on harmless prompts.

1. Find the direction

Harmful and harmless prompt activations form two clusters; r points from the harmless mean to the harmful mean.rharmful promptsharmless prompts

2. Remove it

An activation x is split into its component along the refusal direction and the rest; abliteration keeps only the rest, x prime.r̂everything elsexx′ (kept)removed
Left: each dot is one prompt's activation at a single layer, squashed to 2D. The refusal direction r is the gap between the two means. Right: abliteration splits any activation x into its part along r̂ (r scaled to length 1) and the rest, and keeps only the rest, x′. In a real model "everything else" is thousands of dimensions, not one.

Removing the direction means taking away each activation's component along r̂, the unit-length version of r:

x′ = x − r̂ r̂Tx

You could do this at inference time, with a hook at every layer, but abliteration bakes it into the weights instead. Every matrix that writes to the residual stream (the token embeddings, and the output projections of every attention and MLP block) gets the same treatment, W′ = W − r̂ r̂ᵀW, so no part of the model can write along r̂ any longer. The paper shows the weight edit is equivalent to erasing the direction at run time. What you get is an ordinary checkpoint that complies with requests it would normally decline, and needs no extra code to run.

What it costs

The cost to the model's other abilities is small, but not zero. Arditi et al. found most of their abliterated models scored within noise of the originals on MMLU, ARC and GSM8K, but TruthfulQA scores consistently dropped. Maxime Labonne's write-up abliterated an 8B Llama 3 model and saw a drop on every benchmark run. A round of DPO afterwards won most of it back, except on GSM8K.

The bigger point is about safety. If refusal lives in one direction, the safety training on an open-weight model can be undone with a few hundred prompts and some matrix multiplications, with no fine-tuning data and no training run. As a result, abliterated versions of popular open models are easy to find on Hugging Face. The same property makes abliteration a handy interpretability tool, because it gives a clean, causal test that one direction does what you think it does.

Beyond refusal

Nothing in the recipe is specific to refusal. Swap the harmful and harmless prompts for any two sets that differ in one behaviour and you get a direction for that behaviour, which you can project out the same way. It also works the other way. Arditi et al. made models refuse harmless requests by adding the refusal direction, and Panickssery et al. (2024) used the same difference-of-means trick to turn sycophancy, hallucination and other behaviours up or down in Llama 2, with only a small drop in MMLU. Persona vectors take this further, building directions for traits like sycophancy and "evil" from just a written description of the trait. The catch is that it only works as well as one direction captures the behaviour. Even refusal turns out to be spread over several directions (Wollschläger et al., 2025), and Tan et al. (2024) found steering works well on some inputs and barely at all on others.

References