dorsal/arxiv
View SchemaProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
| Authors | Tao Liu, Taiqiang Wu, Runming Yang, Shaoning Sun, Junjie Wang, Yujiu Yang |
|---|---|
| Categories | |
| ArXiv ID | 2601.09195vv1 |
| URL | https://arxiv.org/abs/2601.09195 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.
{
"annotation_id": "fbfac27b-ca05-4d32-ae30-9887116dffce",
"date_created": "2026-02-17T05:53:20.215000Z",
"date_modified": "2026-02-17T05:53:20.215000Z",
"file_hash": "2f996d2f47e291d347819044acfc11917ba6155388ea96201ec23bbc2f2697a0",
"private": false,
"record": {
"abstract": "Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.",
"arxiv_id": "2601.09195",
"authors": [
"Tao Liu",
"Taiqiang Wu",
"Runming Yang",
"Shaoning Sun",
"Junjie Wang",
"Yujiu Yang"
],
"categories": [
"cs.CL",
"cs.AI"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection",
"url": "https://arxiv.org/abs/2601.09195",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "73864f52-9138-466a-a653-63a3f7d29437",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}