dorsal/arxiv
View SchemaUnleashing the Capabilities of Large Vision-Language Models for Intelligent Perception of Roadside Infrastructure
| Authors | Luxuan Fu, Chong Liu, Bisheng Yang, Zhen Dong |
|---|---|
| Categories | |
| ArXiv ID | 2601.10551vv1 |
| URL | https://arxiv.org/abs/2601.10551 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Automated perception of urban roadside infrastructure is crucial for smart city management, yet general-purpose models often struggle to capture the necessary fine-grained attributes and domain rules. While Large Vision Language Models (VLMs) excel at open-world recognition, they often struggle to accurately interpret complex facility states in compliance with engineering standards, leading to unreliable performance in real-world applications. To address this, we propose a domain-adapted framework that transforms VLMs into specialized agents for intelligent infrastructure analysis. Our approach integrates a data-efficient fine-tuning strategy with a knowledge-grounded reasoning mechanism. Specifically, we leverage open-vocabulary fine-tuning on Grounding DINO to robustly localize diverse assets with minimal supervision, followed by LoRA-based adaptation on Qwen-VL for deep semantic attribute reasoning. To mitigate hallucinations and enforce professional compliance, we introduce a dual-modality Retrieval-Augmented Generation (RAG) module that dynamically retrieves authoritative industry standards and visual exemplars during inference. Evaluated on a comprehensive new dataset of urban roadside scenes, our framework achieves a detection performance of 58.9 mAP and an attribute recognition accuracy of 95.5%, demonstrating a robust solution for intelligent infrastructure monitoring.
{
"annotation_id": "d2cff69a-fb63-47d0-8389-a668c9d1354c",
"date_created": "2026-02-17T05:53:24.389000Z",
"date_modified": "2026-02-17T05:53:24.389000Z",
"file_hash": "20290aae088045c19be1f1cb429d2d8626efbecbdb87459726c777cd0e71041a",
"private": false,
"record": {
"abstract": "Automated perception of urban roadside infrastructure is crucial for smart city management, yet general-purpose models often struggle to capture the necessary fine-grained attributes and domain rules. While Large Vision Language Models (VLMs) excel at open-world recognition, they often struggle to accurately interpret complex facility states in compliance with engineering standards, leading to unreliable performance in real-world applications. To address this, we propose a domain-adapted framework that transforms VLMs into specialized agents for intelligent infrastructure analysis. Our approach integrates a data-efficient fine-tuning strategy with a knowledge-grounded reasoning mechanism. Specifically, we leverage open-vocabulary fine-tuning on Grounding DINO to robustly localize diverse assets with minimal supervision, followed by LoRA-based adaptation on Qwen-VL for deep semantic attribute reasoning. To mitigate hallucinations and enforce professional compliance, we introduce a dual-modality Retrieval-Augmented Generation (RAG) module that dynamically retrieves authoritative industry standards and visual exemplars during inference. Evaluated on a comprehensive new dataset of urban roadside scenes, our framework achieves a detection performance of 58.9 mAP and an attribute recognition accuracy of 95.5%, demonstrating a robust solution for intelligent infrastructure monitoring.",
"arxiv_id": "2601.10551",
"authors": [
"Luxuan Fu",
"Chong Liu",
"Bisheng Yang",
"Zhen Dong"
],
"categories": [
"cs.CV"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Unleashing the Capabilities of Large Vision-Language Models for Intelligent Perception of Roadside Infrastructure",
"url": "https://arxiv.org/abs/2601.10551",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "63102a70-0880-4972-8c4c-1ed1f4c78822",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}