dorsal/arxiv
View SchemaParallel Context-of-Experts Decoding for Retrieval Augmented Generation
| Authors | Giulio Corallo, Paolo Papotti |
|---|---|
| Categories | |
| ArXiv ID | 2601.08670vv1 |
| URL | https://arxiv.org/abs/2601.08670 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Retrieval Augmented Generation faces a trade-off: concatenating documents in a long prompt enables multi-document reasoning but creates prefill bottlenecks, while encoding document KV caches separately offers speed but breaks cross-document interaction. We propose Parallel Context-of-Experts Decoding (Pced), a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding. Pced treats retrieved documents as isolated "experts", synchronizing their predictions via a novel retrieval-aware contrastive decoding rule that weighs expert logits against the model prior. This approach recovers cross-document reasoning capabilities without constructing a shared attention across documents.
{
"annotation_id": "1d7877ec-9100-40d2-8d9d-545b44c9f901",
"date_created": "2026-02-17T05:53:15.286000Z",
"date_modified": "2026-02-17T05:53:15.286000Z",
"file_hash": "6e7bd91a6241025e9c60855f7dab93a2e0764566237b53e50a8c82ec07d8ed96",
"private": false,
"record": {
"abstract": "Retrieval Augmented Generation faces a trade-off: concatenating documents in a long prompt enables multi-document reasoning but creates prefill bottlenecks, while encoding document KV caches separately offers speed but breaks cross-document interaction. We propose Parallel Context-of-Experts Decoding (Pced), a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding. Pced treats retrieved documents as isolated \"experts\", synchronizing their predictions via a novel retrieval-aware contrastive decoding rule that weighs expert logits against the model prior. This approach recovers cross-document reasoning capabilities without constructing a shared attention across documents.",
"arxiv_id": "2601.08670",
"authors": [
"Giulio Corallo",
"Paolo Papotti"
],
"categories": [
"cs.AI",
"cs.CL"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Parallel Context-of-Experts Decoding for Retrieval Augmented Generation",
"url": "https://arxiv.org/abs/2601.08670",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "71dce9e3-8318-4791-8c07-b818f4c547db",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}