dorsal/arxiv
View SchemaLessons from the Field: An Adaptable Lifecycle Approach to Applied Dialogue Summarization
| Authors | Kushal Chawla, Chenyang Zhu, Pengshan Cai, Sangwoo Cho, Scott Novotney, Ayushman Singh, Jonah Lewis, Keasha Safewright, Alfy Samuel, Erin Babinsky, Shi-Xiong Zhang, Sambit Sahu |
|---|---|
| Categories | |
| ArXiv ID | 2601.08682vv1 |
| URL | https://arxiv.org/abs/2601.08682 |
| License | http://creativecommons.org/licenses/by-nc-sa/4.0/ |
Abstract
Summarization of multi-party dialogues is a critical capability in industry, enhancing knowledge transfer and operational effectiveness across many domains. However, automatically generating high-quality summaries is challenging, as the ideal summary must satisfy a set of complex, multi-faceted requirements. While summarization has received immense attention in research, prior work has primarily utilized static datasets and benchmarks, a condition rare in practical scenarios where requirements inevitably evolve. In this work, we present an industry case study on developing an agentic system to summarize multi-party interactions. We share practical insights spanning the full development lifecycle to guide practitioners in building reliable, adaptable summarization systems, as well as to inform future research, covering: 1) robust methods for evaluation despite evolving requirements and task subjectivity, 2) component-wise optimization enabled by the task decomposition inherent in an agentic architecture, 3) the impact of upstream data bottlenecks, and 4) the realities of vendor lock-in due to the poor transferability of LLM prompts.
{
"annotation_id": "5d8e8650-2a40-475b-82bd-2e6cb2378c7f",
"date_created": "2026-02-17T05:53:15.297000Z",
"date_modified": "2026-02-17T05:53:15.297000Z",
"file_hash": "0ded801ee1a678ebe0fe8ec74bdd067e60864efa39fba07ced783768a64f4bc9",
"private": false,
"record": {
"abstract": "Summarization of multi-party dialogues is a critical capability in industry, enhancing knowledge transfer and operational effectiveness across many domains. However, automatically generating high-quality summaries is challenging, as the ideal summary must satisfy a set of complex, multi-faceted requirements. While summarization has received immense attention in research, prior work has primarily utilized static datasets and benchmarks, a condition rare in practical scenarios where requirements inevitably evolve. In this work, we present an industry case study on developing an agentic system to summarize multi-party interactions. We share practical insights spanning the full development lifecycle to guide practitioners in building reliable, adaptable summarization systems, as well as to inform future research, covering: 1) robust methods for evaluation despite evolving requirements and task subjectivity, 2) component-wise optimization enabled by the task decomposition inherent in an agentic architecture, 3) the impact of upstream data bottlenecks, and 4) the realities of vendor lock-in due to the poor transferability of LLM prompts.",
"arxiv_id": "2601.08682",
"authors": [
"Kushal Chawla",
"Chenyang Zhu",
"Pengshan Cai",
"Sangwoo Cho",
"Scott Novotney",
"Ayushman Singh",
"Jonah Lewis",
"Keasha Safewright",
"Alfy Samuel",
"Erin Babinsky",
"Shi-Xiong Zhang",
"Sambit Sahu"
],
"categories": [
"cs.CL",
"cs.AI"
],
"license": "http://creativecommons.org/licenses/by-nc-sa/4.0/",
"title": "Lessons from the Field: An Adaptable Lifecycle Approach to Applied Dialogue Summarization",
"url": "https://arxiv.org/abs/2601.08682",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "7447981e-602c-4641-bb48-c507af270ace",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}