dorsal/arxiv
View SchemaStudent Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration
| Authors | Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li |
|---|---|
| Categories | |
| ArXiv ID | 2601.06160vv1 |
| URL | https://arxiv.org/abs/2601.06160 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
While Large Language Models (LLMs) demonstrate near-human capabilities, they often suffer from "Reasoning Collapse" in complex mathematical proving and long-horizon planning. Models tend to degenerate into low-rank Bias Manifold, where stochastic sampling merely produces lexical variations of erroneous logic rather than semantic exploration. This geometric collapse renders the model "blind" to high-value solutions that lie within its Null Space. To address this, we propose Spectral Orthogonal Exploration (SOE), a geometric framework operating on a counter-intuitive "Student Guides Teacher" paradigm. Specifically, we utilize a weak auxiliary agent not for imitation, but as an orthogonal probe. By explicitly navigating the Teacher's Null Space, SOE serves as a geometric bridge, effectively ejecting the model from local optima to explore diverse, high-value solution spaces. Experiments on mathematical benchmarks demonstrate that, relative to baseline methods, our approach improves average accuracy by 62.4% and increases average sampling efficiency by 113.7%, indicating a promising path toward overcoming performance plateaus in advanced reasoning tasks.
{
"annotation_id": "64b27fe5-ed12-4a1c-bb35-5bd1afd49a57",
"date_created": "2026-02-17T05:53:05.116000Z",
"date_modified": "2026-02-17T05:53:05.116000Z",
"file_hash": "7e74215117f39c3c92c1436b4e1fe547b6cfaa6d80001d23b0ebe486c2897bc8",
"private": false,
"record": {
"abstract": "While Large Language Models (LLMs) demonstrate near-human capabilities, they often suffer from \"Reasoning Collapse\" in complex mathematical proving and long-horizon planning. Models tend to degenerate into low-rank Bias Manifold, where stochastic sampling merely produces lexical variations of erroneous logic rather than semantic exploration. This geometric collapse renders the model \"blind\" to high-value solutions that lie within its Null Space. To address this, we propose Spectral Orthogonal Exploration (SOE), a geometric framework operating on a counter-intuitive \"Student Guides Teacher\" paradigm. Specifically, we utilize a weak auxiliary agent not for imitation, but as an orthogonal probe. By explicitly navigating the Teacher\u0027s Null Space, SOE serves as a geometric bridge, effectively ejecting the model from local optima to explore diverse, high-value solution spaces. Experiments on mathematical benchmarks demonstrate that, relative to baseline methods, our approach improves average accuracy by 62.4% and increases average sampling efficiency by 113.7%, indicating a promising path toward overcoming performance plateaus in advanced reasoning tasks.",
"arxiv_id": "2601.06160",
"authors": [
"Dayu Wang",
"Jiaye Yang",
"Weikang Li",
"Jiahui Liang",
"Yang Li"
],
"categories": [
"cs.AI"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration",
"url": "https://arxiv.org/abs/2601.06160",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "5ee08435-0acc-4790-ae44-30422b642186",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}