dorsal/arxiv
View SchemaFrom Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP
| Authors | Adithya V Ganesan, Vasudha Varadarajan, Oscar NE Kjell, Whitney R Ringwald, Scott Feltman, Benjamin J Luft, Roman Kotov, Ryan L Boyd, H Andrew Schwartz |
|---|---|
| Categories | |
| ArXiv ID | 2601.07988vv1 |
| URL | https://arxiv.org/abs/2601.07988 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
While NLP typically treats documents as independent and unordered samples, in longitudinal studies, this assumption rarely holds: documents are nested within authors and ordered in time, forming person-indexed, time-ordered $\textit{behavioral sequences}$. Here, we demonstrate the need for and propose a longitudinal modeling and evaluation paradigm that consequently updates four parts of the NLP pipeline: (1) evaluation splits aligned to generalization over people ($\textit{cross-sectional}$) and/or time ($\textit{prospective}$); (2) accuracy metrics separating between-person differences from within-person dynamics; (3) sequence inputs to incorporate history by default; and (4) model internals that support different $\textit{coarseness}$ of latent state over histories (pooled summaries, explicit dynamics, or interaction-based models). We demonstrate the issues ensued by traditional pipeline and our proposed improvements on a dataset of 17k daily diary transcripts paired with PTSD symptom severity from 238 participants, finding that traditional document-level evaluation can yield substantially different and sometimes reversed conclusions compared to our ecologically valid modeling and evaluation. We tie our results to a broader discussion motivating a shift from word-sequence evaluation toward $\textit{behavior-sequence}$ paradigms for NLP.
{
"annotation_id": "71dae733-d790-489a-a681-40e4c0817d4b",
"date_created": "2026-02-17T05:53:15.039000Z",
"date_modified": "2026-02-17T05:53:15.039000Z",
"file_hash": "0ff4abb0e7fb82194f811f250d2bd5e2af4201d06a685806f11f614539618b8a",
"private": false,
"record": {
"abstract": "While NLP typically treats documents as independent and unordered samples, in longitudinal studies, this assumption rarely holds: documents are nested within authors and ordered in time, forming person-indexed, time-ordered $\\textit{behavioral sequences}$. Here, we demonstrate the need for and propose a longitudinal modeling and evaluation paradigm that consequently updates four parts of the NLP pipeline: (1) evaluation splits aligned to generalization over people ($\\textit{cross-sectional}$) and/or time ($\\textit{prospective}$); (2) accuracy metrics separating between-person differences from within-person dynamics; (3) sequence inputs to incorporate history by default; and (4) model internals that support different $\\textit{coarseness}$ of latent state over histories (pooled summaries, explicit dynamics, or interaction-based models). We demonstrate the issues ensued by traditional pipeline and our proposed improvements on a dataset of 17k daily diary transcripts paired with PTSD symptom severity from 238 participants, finding that traditional document-level evaluation can yield substantially different and sometimes reversed conclusions compared to our ecologically valid modeling and evaluation. We tie our results to a broader discussion motivating a shift from word-sequence evaluation toward $\\textit{behavior-sequence}$ paradigms for NLP.",
"arxiv_id": "2601.07988",
"authors": [
"Adithya V Ganesan",
"Vasudha Varadarajan",
"Oscar NE Kjell",
"Whitney R Ringwald",
"Scott Feltman",
"Benjamin J Luft",
"Roman Kotov",
"Ryan L Boyd",
"H Andrew Schwartz"
],
"categories": [
"cs.CL",
"cs.AI",
"cs.LG"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP",
"url": "https://arxiv.org/abs/2601.07988",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "b71f2e69-0a66-4958-9d2e-7c5150a171db",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}