dorsal/arxiv
View SchemaRepresentation-Aware Unlearning via Activation Signatures: From Suppression to Knowledge-Signature Erasure
| Authors | Syed Naveed Mahmood, Md. Rezaur Rahman Bhuiyan, Tasfia Zaman, Jareen Tasneem Khondaker, Md. Sameer Sakib, Nazia Tasnim, Farig Sadeque |
|---|---|
| Categories | |
| ArXiv ID | 2601.10566vv2 |
| URL | https://arxiv.org/abs/2601.10566 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Selective knowledge erasure from LLMs is critical for GDPR compliance and model safety, yet current unlearning methods conflate behavioral suppression with true knowledge removal, allowing latent capabilities to persist beneath surface-level refusals. In this work, we address this challenge by introducing Knowledge Immunization Framework (KIF), a representation-aware architecture that distinguishes genuine erasure from obfuscation by targeting internal activation signatures rather than surface outputs. Our approach combines dynamic suppression of subject-specific representations with parameter-efficient adaptation, enabling durable unlearning without full model retraining. KIF achieves near-oracle erasure (FQ approx 0.99 vs. 1.00) while preserving utility at oracle levels (MU = 0.62), effectively breaking the stability-erasure tradeoff that has constrained all prior work. We evaluate both standard foundation models (Llama and Mistral) and reasoning-prior models (Qwen and DeepSeek) across 3B to 14B parameters. Our observation shows that standard models exhibit scale-independent true erasure (<3% utility drift), while reasoning-prior models reveal fundamental architectural divergence. Our comprehensive dual-metric evaluation protocol, combining surface-level leakage with latent trace persistence, operationalizes the obfuscation - erasure distinction and enables the first systematic diagnosis of mechanism-level forgetting behavior across model families and scales.
{
"annotation_id": "0749c202-5896-4c91-9cf8-21c557fccde9",
"date_created": "2026-02-17T05:53:24.065000Z",
"date_modified": "2026-02-17T05:53:24.065000Z",
"file_hash": "6e804bcf8cdeae3fef8279dc592d47c12b5946fea49b1be00f068e8e13ca5410",
"private": false,
"record": {
"abstract": "Selective knowledge erasure from LLMs is critical for GDPR compliance and model safety, yet current unlearning methods conflate behavioral suppression with true knowledge removal, allowing latent capabilities to persist beneath surface-level refusals. In this work, we address this challenge by introducing Knowledge Immunization Framework (KIF), a representation-aware architecture that distinguishes genuine erasure from obfuscation by targeting internal activation signatures rather than surface outputs. Our approach combines dynamic suppression of subject-specific representations with parameter-efficient adaptation, enabling durable unlearning without full model retraining. KIF achieves near-oracle erasure (FQ approx 0.99 vs. 1.00) while preserving utility at oracle levels (MU = 0.62), effectively breaking the stability-erasure tradeoff that has constrained all prior work. We evaluate both standard foundation models (Llama and Mistral) and reasoning-prior models (Qwen and DeepSeek) across 3B to 14B parameters. Our observation shows that standard models exhibit scale-independent true erasure (\u003c3% utility drift), while reasoning-prior models reveal fundamental architectural divergence. Our comprehensive dual-metric evaluation protocol, combining surface-level leakage with latent trace persistence, operationalizes the obfuscation - erasure distinction and enables the first systematic diagnosis of mechanism-level forgetting behavior across model families and scales.",
"arxiv_id": "2601.10566",
"authors": [
"Syed Naveed Mahmood",
"Md. Rezaur Rahman Bhuiyan",
"Tasfia Zaman",
"Jareen Tasneem Khondaker",
"Md. Sameer Sakib",
"Nazia Tasnim",
"Farig Sadeque"
],
"categories": [
"cs.CL",
"cs.LG"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Representation-Aware Unlearning via Activation Signatures: From Suppression to Knowledge-Signature Erasure",
"url": "https://arxiv.org/abs/2601.10566",
"version": "v2"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "a5c44741-7f6c-4566-ac31-16cc712bdb0c",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}