dorsal/arxiv
View SchemaToward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering
| Authors | Xinyu Zhu, Yuzhu Cai, Zexi Liu, Bingyang Zheng, Cheng Wang, Rui Ye, Jiaao Chen, Hanrui Wang, Wei-Chen Wang, Yuzhi Zhang, Linfeng Zhang, Weinan E, Di Jin, Siheng Chen |
|---|---|
| Categories | |
| ArXiv ID | 2601.10402vv2 |
| URL | https://arxiv.org/abs/2601.10402 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles spanning days or weeks. While Large Language Models (LLMs) have demonstrated prowess in short-horizon reasoning, they are easily overwhelmed by execution details in the high-dimensional, delayed-feedback environments of real-world research, failing to consolidate sparse feedback into coherent long-term guidance. Here, we present ML-Master 2.0, an autonomous agent that masters ultra-long-horizon machine learning engineering (MLE) which is a representative microcosm of scientific discovery. By reframing context management as a process of cognitive accumulation, our approach introduces Hierarchical Cognitive Caching (HCC), a multi-tiered architecture inspired by computer systems that enables the structural differentiation of experience over time. By dynamically distilling transient execution traces into stable knowledge and cross-task wisdom, HCC allows agents to decouple immediate execution from long-term experimental strategy, effectively overcoming the scaling limits of static context windows. In evaluations on OpenAI's MLE-Bench under 24-hour budgets, ML-Master 2.0 achieves a state-of-the-art medal rate of 56.44%. Our findings demonstrate that ultra-long-horizon autonomy provides a scalable blueprint for AI capable of autonomous exploration beyond human-precedent complexities.
{
"annotation_id": "82aa670d-17b8-4c8b-8d83-079eef0c6def",
"date_created": "2026-02-17T05:53:24.173000Z",
"date_modified": "2026-02-17T05:53:24.173000Z",
"file_hash": "e5bcd7c08fc2caef12ed9cccb5479038c9e62bba2b0917528701effa7864f3f7",
"private": false,
"record": {
"abstract": "The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles spanning days or weeks. While Large Language Models (LLMs) have demonstrated prowess in short-horizon reasoning, they are easily overwhelmed by execution details in the high-dimensional, delayed-feedback environments of real-world research, failing to consolidate sparse feedback into coherent long-term guidance. Here, we present ML-Master 2.0, an autonomous agent that masters ultra-long-horizon machine learning engineering (MLE) which is a representative microcosm of scientific discovery. By reframing context management as a process of cognitive accumulation, our approach introduces Hierarchical Cognitive Caching (HCC), a multi-tiered architecture inspired by computer systems that enables the structural differentiation of experience over time. By dynamically distilling transient execution traces into stable knowledge and cross-task wisdom, HCC allows agents to decouple immediate execution from long-term experimental strategy, effectively overcoming the scaling limits of static context windows. In evaluations on OpenAI\u0027s MLE-Bench under 24-hour budgets, ML-Master 2.0 achieves a state-of-the-art medal rate of 56.44%. Our findings demonstrate that ultra-long-horizon autonomy provides a scalable blueprint for AI capable of autonomous exploration beyond human-precedent complexities.",
"arxiv_id": "2601.10402",
"authors": [
"Xinyu Zhu",
"Yuzhu Cai",
"Zexi Liu",
"Bingyang Zheng",
"Cheng Wang",
"Rui Ye",
"Jiaao Chen",
"Hanrui Wang",
"Wei-Chen Wang",
"Yuzhi Zhang",
"Linfeng Zhang",
"Weinan E",
"Di Jin",
"Siheng Chen"
],
"categories": [
"cs.AI"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering",
"url": "https://arxiv.org/abs/2601.10402",
"version": "v2"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "d0010ca8-959d-4759-a363-248dad9c6b2d",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}