dorsal/arxiv
View SchemaNewsScope: Schema-Grounded Cross-Domain News Claim Extraction with Open Models
| Authors | Nidhi Pandya |
|---|---|
| Categories | |
| ArXiv ID | 2601.08852vv1 |
| URL | https://arxiv.org/abs/2601.08852 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Automated news verification requires structured claim extraction, but existing approaches either lack schema compliance or generalize poorly across domains. This paper presents NewsScope, a cross-domain dataset, benchmark, and fine-tuned model for schema-grounded news claim extraction. The dataset contains 455 articles across politics, health, science/environment, and business, consisting of 395 in-domain articles and 60 out-of-source articles for generalization testing. LLaMA 3.1 8B was fine-tuned using LoRA on 315 training examples and evaluated on held-out in-domain (80 articles) and out-of-source (60 articles) test sets. Human evaluation on 400 claims shows NewsScope achieves 89.4% human-evaluated accuracy compared to GPT-4o-mini's 93.7% (p=0.07). NewsScope outperforms GPT-4o-mini on political claims (94.3% vs. 87.8%). A numeric grounding filter further improves accuracy to 91.6%, narrowing the gap to 2.1 percentage points. Inter-annotator agreement studies (160 claims) confirm labeling reliability (94.6% positive agreement on SUPPORTED judgments). The open-weight model enables offline deployment at approximately $15 on-demand compute (or $0 on free tiers). Code and benchmark are publicly released.
{
"annotation_id": "04849277-b7a2-4050-8e52-bd75f4d796dd",
"date_created": "2026-02-17T05:53:19.999000Z",
"date_modified": "2026-02-17T05:53:19.999000Z",
"file_hash": "a2f251eee2b76210dffd5f5cf99f1382c2ff79b4f6b54282cd77dcf2e8177767",
"private": false,
"record": {
"abstract": "Automated news verification requires structured claim extraction, but existing approaches either lack schema compliance or generalize poorly across domains. This paper presents NewsScope, a cross-domain dataset, benchmark, and fine-tuned model for schema-grounded news claim extraction. The dataset contains 455 articles across politics, health, science/environment, and business, consisting of 395 in-domain articles and 60 out-of-source articles for generalization testing. LLaMA 3.1 8B was fine-tuned using LoRA on 315 training examples and evaluated on held-out in-domain (80 articles) and out-of-source (60 articles) test sets. Human evaluation on 400 claims shows NewsScope achieves 89.4% human-evaluated accuracy compared to GPT-4o-mini\u0027s 93.7% (p=0.07). NewsScope outperforms GPT-4o-mini on political claims (94.3% vs. 87.8%). A numeric grounding filter further improves accuracy to 91.6%, narrowing the gap to 2.1 percentage points. Inter-annotator agreement studies (160 claims) confirm labeling reliability (94.6% positive agreement on SUPPORTED judgments). The open-weight model enables offline deployment at approximately $15 on-demand compute (or $0 on free tiers). Code and benchmark are publicly released.",
"arxiv_id": "2601.08852",
"authors": [
"Nidhi Pandya"
],
"categories": [
"cs.CL"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "NewsScope: Schema-Grounded Cross-Domain News Claim Extraction with Open Models",
"url": "https://arxiv.org/abs/2601.08852",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "1abdbb86-730f-4225-ae82-ffccb06a5bb6",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}