dorsal/arxiv
View SchemaJudgeFlow: Agentic Workflow Optimization via Block Judge
| Authors | Zihan Ma, Zhikai Zhao, Chuanbo Hua, Federico Berto, Jinkyoo Park |
|---|---|
| Categories | |
| ArXiv ID | 2601.07477vv1 |
| URL | https://arxiv.org/abs/2601.07477 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Optimizing LLM-based agentic workflows is challenging for scaling AI capabilities. Current methods rely on coarse, end-to-end evaluation signals and lack fine-grained signals on where to refine, often resulting in inefficient or low-impact modifications. To address these limitations, we propose {\our{}}, an Evaluation-Judge-Optimization-Update pipeline. We incorporate reusable, configurable logic blocks into agentic workflows to capture fundamental forms of logic. On top of this abstraction, we design a dedicated Judge module that inspects execution traces -- particularly failed runs -- and assigns rank-based responsibility scores to problematic blocks. These fine-grained diagnostic signals are then leveraged by an LLM-based optimizer, which focuses modifications on the most problematic block in the workflow. Our approach improves sample efficiency, enhances interpretability through block-level diagnostics, and provides a scalable foundation for automating increasingly complex agentic workflows. We evaluate {\our{}} on mathematical reasoning and code generation benchmarks, where {\our{}} achieves superior performance and efficiency compared to existing methods. The source code is publicly available at https://github.com/ma-zihan/JudgeFlow.
{
"annotation_id": "633d4ba5-f4e6-4aec-afd1-2ab1d00a4d06",
"date_created": "2026-02-17T05:53:12.391000Z",
"date_modified": "2026-02-17T05:53:12.391000Z",
"file_hash": "20726e7320057c9a3f287b36da37034eb447b1e87d519a4baa4143409a6bd43a",
"private": false,
"record": {
"abstract": "Optimizing LLM-based agentic workflows is challenging for scaling AI capabilities. Current methods rely on coarse, end-to-end evaluation signals and lack fine-grained signals on where to refine, often resulting in inefficient or low-impact modifications. To address these limitations, we propose {\\our{}}, an Evaluation-Judge-Optimization-Update pipeline. We incorporate reusable, configurable logic blocks into agentic workflows to capture fundamental forms of logic. On top of this abstraction, we design a dedicated Judge module that inspects execution traces -- particularly failed runs -- and assigns rank-based responsibility scores to problematic blocks. These fine-grained diagnostic signals are then leveraged by an LLM-based optimizer, which focuses modifications on the most problematic block in the workflow. Our approach improves sample efficiency, enhances interpretability through block-level diagnostics, and provides a scalable foundation for automating increasingly complex agentic workflows. We evaluate {\\our{}} on mathematical reasoning and code generation benchmarks, where {\\our{}} achieves superior performance and efficiency compared to existing methods. The source code is publicly available at https://github.com/ma-zihan/JudgeFlow.",
"arxiv_id": "2601.07477",
"authors": [
"Zihan Ma",
"Zhikai Zhao",
"Chuanbo Hua",
"Federico Berto",
"Jinkyoo Park"
],
"categories": [
"cs.AI"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "JudgeFlow: Agentic Workflow Optimization via Block Judge",
"url": "https://arxiv.org/abs/2601.07477",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "a57c054f-cc15-4136-aa2d-90b3f32c2a89",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}