dorsal/arxiv
View SchemaCollaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning
| Authors | Zhiyuan Hu, Yunhai Hu, Juncheng Liu, Shuyue Stella Li, Yucheng Wang, Zhen Xu, See-Kiong Ng, Anh Tuan Luu, Xinxing Xu, Bryan Hooi, Cynthia Breazeal, Hae Won Park |
|---|---|
| Categories | |
| ArXiv ID | 2601.09667vv2 |
| URL | https://arxiv.org/abs/2601.09667 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Multi-agent systems have evolved into practical LLM-driven collaborators for many applications, gaining robustness from diversity and cross-checking. However, multi-agent RL (MARL) training is resource-intensive and unstable: co-adapting teammates induce non-stationarity, and rewards are often sparse and high-variance. Therefore, we introduce \textbf{Multi-Agent Test-Time Reinforcement Learning (MATTRL)}, a framework that injects structured textual experience into multi-agent deliberation at inference time. MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making. We also study credit assignment for constructing a turn-level experience pool, then reinjecting it into the dialogue. Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67\% over a multi-agent baseline, and by 8.67\% over comparable single-agent baselines. Ablation studies examine different credit-assignment schemes and provide a detailed comparison of how they affect training outcomes. MATTRL offers a stable, effective and efficient path to distribution-shift-robust multi-agent reasoning without tuning.
{
"annotation_id": "51d4490f-79ab-4093-bfaf-309b5c14a56c",
"date_created": "2026-02-17T05:53:20.408000Z",
"date_modified": "2026-02-17T05:53:20.408000Z",
"file_hash": "931dc5b098346ee2ed838ab6096b69cbfe83578714b103772a7f37a3bcb2ecd8",
"private": false,
"record": {
"abstract": "Multi-agent systems have evolved into practical LLM-driven collaborators for many applications, gaining robustness from diversity and cross-checking. However, multi-agent RL (MARL) training is resource-intensive and unstable: co-adapting teammates induce non-stationarity, and rewards are often sparse and high-variance. Therefore, we introduce \\textbf{Multi-Agent Test-Time Reinforcement Learning (MATTRL)}, a framework that injects structured textual experience into multi-agent deliberation at inference time. MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making. We also study credit assignment for constructing a turn-level experience pool, then reinjecting it into the dialogue. Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67\\% over a multi-agent baseline, and by 8.67\\% over comparable single-agent baselines. Ablation studies examine different credit-assignment schemes and provide a detailed comparison of how they affect training outcomes. MATTRL offers a stable, effective and efficient path to distribution-shift-robust multi-agent reasoning without tuning.",
"arxiv_id": "2601.09667",
"authors": [
"Zhiyuan Hu",
"Yunhai Hu",
"Juncheng Liu",
"Shuyue Stella Li",
"Yucheng Wang",
"Zhen Xu",
"See-Kiong Ng",
"Anh Tuan Luu",
"Xinxing Xu",
"Bryan Hooi",
"Cynthia Breazeal",
"Hae Won Park"
],
"categories": [
"cs.AI",
"cs.CL"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning",
"url": "https://arxiv.org/abs/2601.09667",
"version": "v2"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "1a3bb692-9751-48ce-a1c2-844f17f0c6eb",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}