dorsal/arxiv
View SchemaHIPPO: Accelerating Video Large Language Models Inference via Holistic-aware Parallel Speculative Decoding
| Authors | Qitan Lv, Tianyu Liu, Wen Wu, Xuenan Xu, Bowen Zhou, Feng Wu, Chao Zhang |
|---|---|
| Categories | |
| ArXiv ID | 2601.08273vv1 |
| URL | https://arxiv.org/abs/2601.08273 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Speculative decoding (SD) has emerged as a promising approach to accelerate LLM inference without sacrificing output quality. Existing SD methods tailored for video-LLMs primarily focus on pruning redundant visual tokens to mitigate the computational burden of massive visual inputs. However, existing methods do not achieve inference acceleration comparable to text-only LLMs. We observe from extensive experiments that this phenomenon mainly stems from two limitations: (i) their pruning strategies inadequately preserve visual semantic tokens, degrading draft quality and acceptance rates; (ii) even with aggressive pruning (e.g., 90% visual tokens removed), the draft model's remaining inference cost limits overall speedup. To address these limitations, we propose HIPPO, a general holistic-aware parallel speculative decoding framework. Specifically, HIPPO proposes (i) a semantic-aware token preservation method, which fuses global attention scores with local visual semantics to retain semantic information at high pruning ratios; (ii) a video parallel SD algorithm that decouples and overlaps draft generation and target verification phases. Experiments on four video-LLMs across six benchmarks demonstrate HIPPO's effectiveness, yielding up to 3.51x speedup compared to vanilla auto-regressive decoding.
{
"annotation_id": "217d3e45-ee46-4aad-9236-0bec85f4ef6f",
"date_created": "2026-02-17T05:53:15.958000Z",
"date_modified": "2026-02-17T05:53:15.958000Z",
"file_hash": "c1059e49eea74a62aff710cbe413b55b4fde09b4d602ad28eea806093f039ada",
"private": false,
"record": {
"abstract": "Speculative decoding (SD) has emerged as a promising approach to accelerate LLM inference without sacrificing output quality. Existing SD methods tailored for video-LLMs primarily focus on pruning redundant visual tokens to mitigate the computational burden of massive visual inputs. However, existing methods do not achieve inference acceleration comparable to text-only LLMs. We observe from extensive experiments that this phenomenon mainly stems from two limitations: (i) their pruning strategies inadequately preserve visual semantic tokens, degrading draft quality and acceptance rates; (ii) even with aggressive pruning (e.g., 90% visual tokens removed), the draft model\u0027s remaining inference cost limits overall speedup. To address these limitations, we propose HIPPO, a general holistic-aware parallel speculative decoding framework. Specifically, HIPPO proposes (i) a semantic-aware token preservation method, which fuses global attention scores with local visual semantics to retain semantic information at high pruning ratios; (ii) a video parallel SD algorithm that decouples and overlaps draft generation and target verification phases. Experiments on four video-LLMs across six benchmarks demonstrate HIPPO\u0027s effectiveness, yielding up to 3.51x speedup compared to vanilla auto-regressive decoding.",
"arxiv_id": "2601.08273",
"authors": [
"Qitan Lv",
"Tianyu Liu",
"Wen Wu",
"Xuenan Xu",
"Bowen Zhou",
"Feng Wu",
"Chao Zhang"
],
"categories": [
"cs.CV",
"cs.AI"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "HIPPO: Accelerating Video Large Language Models Inference via Holistic-aware Parallel Speculative Decoding",
"url": "https://arxiv.org/abs/2601.08273",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "399975c9-80f1-4f6f-8993-bf4ee6ec4eb2",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}