dorsal/arxiv
View SchemaNoReGeo: Non-Reasoning Geometry Benchmark
| Authors | Irina Abdullaeva, Anton Vasiliuk, Elizaveta Goncharova, Temurbek Rahmatullaev, Zagorulko Ivan, Maxim Kurkin, Andrey Kuznetsov |
|---|---|
| Categories | |
| ArXiv ID | 2601.10254vv1 |
| URL | https://arxiv.org/abs/2601.10254 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
We present NoReGeo, a novel benchmark designed to evaluate the intrinsic geometric understanding of large language models (LLMs) without relying on reasoning or algebraic computation. Unlike existing benchmarks that primarily assess models' proficiency in reasoning-based geometry-where solutions are derived using algebraic methods-NoReGeo focuses on evaluating whether LLMs can inherently encode spatial relationships and recognize geometric properties directly. Our benchmark comprises 2,500 trivial geometric problems spanning 25 categories, each carefully crafted to be solvable purely through native geometric understanding, assuming known object locations. We assess a range of state-of-the-art models on NoReGeo, including frontier models like GPT-4, observing that even the most advanced systems achieve an overall maximum of 65% accuracy in binary classification tasks. Further, our ablation experiments demonstrate that such geometric understanding does not emerge through fine-tuning alone, indicating that effective training for geometric comprehension requires a specialized approach from the outset. Our findings highlight a significant gap in current LLMs' ability to natively grasp geometric concepts, providing a foundation for future research toward models with true geometric cognition.
{
"annotation_id": "ba2adaa9-d568-45d7-82c4-0c27cdb24a47",
"date_created": "2026-02-17T05:53:24.221000Z",
"date_modified": "2026-02-17T05:53:24.221000Z",
"file_hash": "e09065053d75b6574820db2a5ad59cbd84900ac22c1ea94451f24eb81a77290b",
"private": false,
"record": {
"abstract": "We present NoReGeo, a novel benchmark designed to evaluate the intrinsic geometric understanding of large language models (LLMs) without relying on reasoning or algebraic computation. Unlike existing benchmarks that primarily assess models\u0027 proficiency in reasoning-based geometry-where solutions are derived using algebraic methods-NoReGeo focuses on evaluating whether LLMs can inherently encode spatial relationships and recognize geometric properties directly. Our benchmark comprises 2,500 trivial geometric problems spanning 25 categories, each carefully crafted to be solvable purely through native geometric understanding, assuming known object locations. We assess a range of state-of-the-art models on NoReGeo, including frontier models like GPT-4, observing that even the most advanced systems achieve an overall maximum of 65% accuracy in binary classification tasks. Further, our ablation experiments demonstrate that such geometric understanding does not emerge through fine-tuning alone, indicating that effective training for geometric comprehension requires a specialized approach from the outset. Our findings highlight a significant gap in current LLMs\u0027 ability to natively grasp geometric concepts, providing a foundation for future research toward models with true geometric cognition.",
"arxiv_id": "2601.10254",
"authors": [
"Irina Abdullaeva",
"Anton Vasiliuk",
"Elizaveta Goncharova",
"Temurbek Rahmatullaev",
"Zagorulko Ivan",
"Maxim Kurkin",
"Andrey Kuznetsov"
],
"categories": [
"cs.AI"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "NoReGeo: Non-Reasoning Geometry Benchmark",
"url": "https://arxiv.org/abs/2601.10254",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "c06bf72c-e994-40f7-ab77-ecb78429855c",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}