Heretic is an open-source tool that automatically removes safety alignment (refusal behavior) from transformer-based language models using directional ablation — without retraining or manual configuration.

How is Heretic different from OBLITERATUS?

OBLITERATUS is a research toolkit that exposes the full abliteration pipeline with manual controls and deep analysis. Heretic is a turnkey solution that uses Optuna to automatically find optimal abliteration parameters — no transformer knowledge required.

Does Heretic damage model capabilities?

Heretic co-minimizes refusals AND KL divergence from the original model, preserving as much of the original intelligence as possible. Benchmarks show Heretic models achieve lower KL divergence than manually abliterated models at the same refusal suppression level.

What models does Heretic support?

Most dense transformer models including Llama, Qwen, Gemma, and multimodal models. It also supports several MoE architectures. State-space models (like Mamba) are not yet supported.

How long does abliteration take with Heretic?

On an RTX 3090, decensoring a 4B parameter model takes 20-30 minutes. Heretic auto-benchmarks your hardware and selects optimal batch sizes. 4-bit quantization is supported for lower VRAM systems.

Heretic: Zero-Config Abliteration Comes to the Command Line

By Prahlad Menon Published 2026-07-12 5 min read

In April, I covered OBLITERATUS — a research toolkit that exposes the geometry of refusal inside language models. It demonstrated that safety alignment via RLHF creates a linear feature: a direction in activation space that you can project out of the weights entirely.

OBLITERATUS is a scalpel. It gives you probes, distillation methods, layer-by-layer analysis, and full control over the surgical removal of refusal directions. Researchers love it.

But most people don’t want a scalpel. They want a button.

Heretic is that button.

One Command, No Configuration

pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507

That’s it. Heretic benchmarks your hardware, determines optimal batch sizes, runs the abliteration pipeline with Optuna-powered hyperparameter search, and outputs a decensored model. No understanding of transformer internals required.

The key innovation is the optimization loop. Previous abliteration tools required manual tuning — which layers to target, what projection strength to use, how to balance refusal removal against capability preservation. Heretic solves this with Optuna, a Tree-structured Parzen Estimator (TPE) optimizer that:

Co-minimizes two objectives: refusal rate (should drop to near-zero) and KL divergence from the original model (should stay as low as possible)
Auto-discovers parameters: layer selection, projection strength, and ablation targets are found through search, not manual specification
Runs unsupervised: default configuration produces results competitive with expert-tuned abliterations

Benchmarks: Better Than Manual

The results speak for themselves. On Gemma 3 12B:

Model	Refusals (out of 100)	KL Divergence
google/gemma-3-12b-it (original)	97	0 (baseline)
mlabonne/gemma-3-12b-it-abliterated-v2	3	1.04
huihui-ai/gemma-3-12b-it-abliterated	3	0.45
p-e-w/gemma-3-12b-it-heretic	3	0.16

Same refusal suppression. One-sixth the divergence from original capabilities. And zero human effort in the tuning process.

The community has published over 4,000 models using Heretic — a testament to how fully the barrier has dropped.

The Abliteration Pipeline, Explained

For those who want to understand what’s happening under the hood, Heretic implements the same core technique as OBLITERATUS:

Collect activations on paired prompt sets — restricted (“harmful”) prompts that trigger refusal, and unrestricted (“harmless”) prompts that don’t
Extract the refusal direction — the consistent vector difference between how the model processes these two categories
Project the direction out of the relevant weight matrices, using norm-preserving methods that minimize collateral damage

The difference is automation. OBLITERATUS exposes every step with full configurability. Heretic wraps the entire pipeline in an optimization loop that finds good parameters automatically.

Both tools build on the same foundational research: Arditi et al. 2024 on refusal as a linear feature, and grimjim’s 2025 work on norm-preserving biprojection.

Research Features

Heretic isn’t just a one-click abliterator. With the research extra (pip install heretic-llm[research]), it provides interpretability visualizations:

PaCMAP projections of residual vectors across layers, showing how “harmful” and “harmless” prompt representations separate and cluster
Layer-by-layer animations of activation geometry
Quantitative metrics on residual vector relationships (cosine similarity, norms, silhouette scores)

This positions Heretic somewhere between a consumer tool and a research platform — accessible enough for anyone to use, but deep enough for mechanistic interpretability work.

What This Means for AI Safety

The uncomfortable reality: if your safety alignment can be removed by running a single command, it was never robust.

But this isn’t news. Abliteration has been possible since mid-2024. What Heretic changes is the effort threshold. Previously, you needed to understand directional ablation, experiment with layer selection, and tune parameters. Now you need to know how to install a pip package.

This matters for several reasons:

Open weights are truly open. Any safety measures baked into Llama, Qwen, Gemma, etc. are advisory, not architectural. Release a model, and the community will abliterate it within hours.
Red-teaming is table stakes. If you’re building applications on open-weight models and assuming refusal behavior will persist, you’re building on sand.
The research community has spoken. 4,000+ abliterated models means the demand for unrestricted local inference is massive. Ignoring this doesn’t make it go away.

The path forward isn’t trying to make abliteration harder. It’s recognizing that shallow alignment — adding a removable direction to activation space — was always a stopgap, not a solution.

Try It

pip install -U heretic-llm
heretic meta-llama/Llama-3.1-8B-Instruct

On an RTX 3090, expect 20-30 minutes for a 4B model. Heretic supports --quantization bnb_4bit for lower VRAM systems.

The project is MIT licensed and actively maintained. Models are automatically tagged with Heretic metadata for reproducibility.

The Bigger Picture

I wrote in the OBLITERATUS post that the real lesson of abliteration isn’t jailbreaking — it’s that linear features in neural networks are both a vulnerability and a window into how these systems work.

Heretic drives that lesson home. When an intern with a command line can undo millions of dollars of alignment work in 30 minutes, the question isn’t whether abliteration is good or bad. It’s: what does robust alignment actually look like?

We don’t have that answer yet. But tools like Heretic and OBLITERATUS are forcing the question.