dorsal/arxiv
View SchemaReCo-KD: Region- and Context-Aware Knowledge Distillation for Efficient 3D Medical Image Segmentation
| Authors | Qizhen Lan, Yu-Chun Hsu, Nida Saddaf Khan, Xiaoqian Jiang |
|---|---|
| Categories | |
| ArXiv ID | 2601.08301vv1 |
| URL | https://arxiv.org/abs/2601.08301 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Accurate 3D medical image segmentation is vital for diagnosis and treatment planning, but state-of-the-art models are often too large for clinics with limited computing resources. Lightweight architectures typically suffer significant performance loss. To address these deployment and speed constraints, we propose Region- and Context-aware Knowledge Distillation (ReCo-KD), a training-only framework that transfers both fine-grained anatomical detail and long-range contextual information from a high-capacity teacher to a compact student network. The framework integrates Multi-Scale Structure-Aware Region Distillation (MS-SARD), which applies class-aware masks and scale-normalized weighting to emphasize small but clinically important regions, and Multi-Scale Context Alignment (MS-CA), which aligns teacher-student affinity patterns across feature levels. Implemented on nnU-Net in a backbone-agnostic manner, ReCo-KD requires no custom student design and is easily adapted to other architectures. Experiments on multiple public 3D medical segmentation datasets and a challenging aggregated dataset show that the distilled lightweight model attains accuracy close to the teacher while markedly reducing parameters and inference latency, underscoring its practicality for clinical deployment.
{
"annotation_id": "35e24c72-22a6-42d7-b056-86688f57857f",
"date_created": "2026-02-17T05:53:15.442000Z",
"date_modified": "2026-02-17T05:53:15.442000Z",
"file_hash": "313b69451cc2a58ecb1d856bc30c53ce58f80253e1f689689d139a3bc6c6127a",
"private": false,
"record": {
"abstract": "Accurate 3D medical image segmentation is vital for diagnosis and treatment planning, but state-of-the-art models are often too large for clinics with limited computing resources. Lightweight architectures typically suffer significant performance loss. To address these deployment and speed constraints, we propose Region- and Context-aware Knowledge Distillation (ReCo-KD), a training-only framework that transfers both fine-grained anatomical detail and long-range contextual information from a high-capacity teacher to a compact student network. The framework integrates Multi-Scale Structure-Aware Region Distillation (MS-SARD), which applies class-aware masks and scale-normalized weighting to emphasize small but clinically important regions, and Multi-Scale Context Alignment (MS-CA), which aligns teacher-student affinity patterns across feature levels. Implemented on nnU-Net in a backbone-agnostic manner, ReCo-KD requires no custom student design and is easily adapted to other architectures. Experiments on multiple public 3D medical segmentation datasets and a challenging aggregated dataset show that the distilled lightweight model attains accuracy close to the teacher while markedly reducing parameters and inference latency, underscoring its practicality for clinical deployment.",
"arxiv_id": "2601.08301",
"authors": [
"Qizhen Lan",
"Yu-Chun Hsu",
"Nida Saddaf Khan",
"Xiaoqian Jiang"
],
"categories": [
"cs.CV"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "ReCo-KD: Region- and Context-Aware Knowledge Distillation for Efficient 3D Medical Image Segmentation",
"url": "https://arxiv.org/abs/2601.08301",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "ef0be73f-6033-4577-b24e-d4b0e7c82f9e",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}