dorsal/arxiv
View SchemaCtrlFuse: Mask-Prompt Guided Controllable Infrared and Visible Image Fusion
| Authors | Yiming Sun, Yuan Ruan, Qinghua Hu, Pengfei Zhu |
|---|---|
| Categories | |
| ArXiv ID | 2601.08619vv1 |
| URL | https://arxiv.org/abs/2601.08619 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Infrared and visible image fusion generates all-weather perception-capable images by combining complementary modalities, enhancing environmental awareness for intelligent unmanned systems. Existing methods either focus on pixel-level fusion while overlooking downstream task adaptability or implicitly learn rigid semantics through cascaded detection/segmentation models, unable to interactively address diverse semantic target perception needs. We propose CtrlFuse, a controllable image fusion framework that enables interactive dynamic fusion guided by mask prompts. The model integrates a multi-modal feature extractor, a reference prompt encoder (RPE), and a prompt-semantic fusion module (PSFM). The RPE dynamically encodes task-specific semantic prompts by fine-tuning pre-trained segmentation models with input mask guidance, while the PSFM explicitly injects these semantics into fusion features. Through synergistic optimization of parallel segmentation and fusion branches, our method achieves mutual enhancement between task performance and fusion quality. Experiments demonstrate state-of-the-art results in both fusion controllability and segmentation accuracy, with the adapted task branch even outperforming the original segmentation model.
{
"annotation_id": "173307a3-c264-434c-94e6-071adbaa9b63",
"date_created": "2026-02-17T05:53:15.285000Z",
"date_modified": "2026-02-17T05:53:15.285000Z",
"file_hash": "a0b5d46ab73811fa7cb3217feacd4c93a50411a1cd82562751c276a7f69b792c",
"private": false,
"record": {
"abstract": "Infrared and visible image fusion generates all-weather perception-capable images by combining complementary modalities, enhancing environmental awareness for intelligent unmanned systems. Existing methods either focus on pixel-level fusion while overlooking downstream task adaptability or implicitly learn rigid semantics through cascaded detection/segmentation models, unable to interactively address diverse semantic target perception needs. We propose CtrlFuse, a controllable image fusion framework that enables interactive dynamic fusion guided by mask prompts. The model integrates a multi-modal feature extractor, a reference prompt encoder (RPE), and a prompt-semantic fusion module (PSFM). The RPE dynamically encodes task-specific semantic prompts by fine-tuning pre-trained segmentation models with input mask guidance, while the PSFM explicitly injects these semantics into fusion features. Through synergistic optimization of parallel segmentation and fusion branches, our method achieves mutual enhancement between task performance and fusion quality. Experiments demonstrate state-of-the-art results in both fusion controllability and segmentation accuracy, with the adapted task branch even outperforming the original segmentation model.",
"arxiv_id": "2601.08619",
"authors": [
"Yiming Sun",
"Yuan Ruan",
"Qinghua Hu",
"Pengfei Zhu"
],
"categories": [
"cs.CV"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "CtrlFuse: Mask-Prompt Guided Controllable Infrared and Visible Image Fusion",
"url": "https://arxiv.org/abs/2601.08619",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "1a2beafa-6996-42bf-84d6-4fda165d2c30",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}