dorsal/arxiv
View SchemaDialogue Telemetry: Turn-Level Instrumentation for Autonomous Information Gathering
| Authors | Dimitris Panagopoulos, Adolfo Perrusquia, Weisi Guo |
|---|---|
| Categories | |
| ArXiv ID | 2601.09570vv1 |
| URL | https://arxiv.org/abs/2601.09570 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Autonomous systems conducting schema-grounded information-gathering dialogues face an instrumentation gap, lacking turn-level observables for monitoring acquisition efficiency and detecting when questioning becomes unproductive. We introduce Dialogue Telemetry (DT), a measurement framework that produces two model-agnostic signals after each question-answer exchange: (i) a Progress Estimator (PE) quantifying residual information potential per category (with a bits-based variant), and (ii) a Stalling Index (SI) detecting an observable failure signature characterized by repeated category probing with semantically similar, low-marginal-gain responses. SI flags this pattern without requiring causal diagnosis, supporting monitoring in settings where attributing degradation to specific causes may be impractical. We validate DT in controlled search-and-rescue (SAR)-inspired interviews using large language model (LLM)-based simulations, distinguishing efficient from stalled dialogue traces and illustrating downstream utility by integrating DT signals into a reinforcement learning (RL) policy. Across these settings, DT provides interpretable turn-level instrumentation that improves policy performance when stalling carries operational costs.
{
"annotation_id": "f80de57f-62bb-401c-a684-f88a7dd1b32d",
"date_created": "2026-02-17T05:53:19.953000Z",
"date_modified": "2026-02-17T05:53:19.953000Z",
"file_hash": "d3f006394927621f33b1c1c9ee5e2c7b943426801998dc926568f31c4fc104b3",
"private": false,
"record": {
"abstract": "Autonomous systems conducting schema-grounded information-gathering dialogues face an instrumentation gap, lacking turn-level observables for monitoring acquisition efficiency and detecting when questioning becomes unproductive. We introduce Dialogue Telemetry (DT), a measurement framework that produces two model-agnostic signals after each question-answer exchange: (i) a Progress Estimator (PE) quantifying residual information potential per category (with a bits-based variant), and (ii) a Stalling Index (SI) detecting an observable failure signature characterized by repeated category probing with semantically similar, low-marginal-gain responses. SI flags this pattern without requiring causal diagnosis, supporting monitoring in settings where attributing degradation to specific causes may be impractical. We validate DT in controlled search-and-rescue (SAR)-inspired interviews using large language model (LLM)-based simulations, distinguishing efficient from stalled dialogue traces and illustrating downstream utility by integrating DT signals into a reinforcement learning (RL) policy. Across these settings, DT provides interpretable turn-level instrumentation that improves policy performance when stalling carries operational costs.",
"arxiv_id": "2601.09570",
"authors": [
"Dimitris Panagopoulos",
"Adolfo Perrusquia",
"Weisi Guo"
],
"categories": [
"cs.CL"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Dialogue Telemetry: Turn-Level Instrumentation for Autonomous Information Gathering",
"url": "https://arxiv.org/abs/2601.09570",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "84c64ce0-9837-436b-a0e4-9ba6fbea1ebd",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}