dorsal/arxiv
View SchemaAntiPaSTO: Self-Supervised Steering of Moral Reasoning
| Authors | Michael J. Clark |
|---|---|
| Categories | |
| ArXiv ID | 2601.07473vv1 |
| URL | https://arxiv.org/abs/2601.07473 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
As models grow more capable, human supervision breaks down: labels don't scale, outputs can be gamed, and training doesn't generalize. Scalable oversight requires steering methods that are internal, self-supervised, and transfer out-of-distribution; existing methods satisfy some but not all three. We introduce AntiPaSTO, which separates representations along an anti-parallel axis ($\alpha=\pm1$ produce opposite shifts), with coherence constraints preventing collapse. Human input is minimal: two contrasting words inserted into template sentences, no preference labels. Using 800 such pairs on Gemma-3-1B, AntiPaSTO beats prompting baselines by $6.9\times$ on DailyDilemmas and maintains bidirectional control where prompting triggers refusal. Code is available at https://github.com/wassname/AntiPaSTO.
{
"annotation_id": "a1f2b10f-733f-4d57-bb7d-ad77aa81ebbd",
"date_created": "2026-02-17T05:53:12.366000Z",
"date_modified": "2026-02-17T05:53:12.366000Z",
"file_hash": "f4a75b38140714b5acafb9acde5f1af5e5d6afef7edf1f72b859e5de3525b77b",
"private": false,
"record": {
"abstract": "As models grow more capable, human supervision breaks down: labels don\u0027t scale, outputs can be gamed, and training doesn\u0027t generalize. Scalable oversight requires steering methods that are internal, self-supervised, and transfer out-of-distribution; existing methods satisfy some but not all three. We introduce AntiPaSTO, which separates representations along an anti-parallel axis ($\\alpha=\\pm1$ produce opposite shifts), with coherence constraints preventing collapse. Human input is minimal: two contrasting words inserted into template sentences, no preference labels. Using 800 such pairs on Gemma-3-1B, AntiPaSTO beats prompting baselines by $6.9\\times$ on DailyDilemmas and maintains bidirectional control where prompting triggers refusal.\n Code is available at https://github.com/wassname/AntiPaSTO.",
"arxiv_id": "2601.07473",
"authors": [
"Michael J. Clark"
],
"categories": [
"cs.LG"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "AntiPaSTO: Self-Supervised Steering of Moral Reasoning",
"url": "https://arxiv.org/abs/2601.07473",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "479aad34-780f-42b0-a6f6-c4fe57e66948",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}