dorsal/arxiv
View SchemaOn the Limits of Self-Improving in LLMs and Why AGI, ASI and the Singularity Are Not Near Without Symbolic Model Synthesis
| Authors | Hector Zenil |
|---|---|
| Categories | |
| ArXiv ID | 2601.05280vv1 |
| URL | https://arxiv.org/abs/2601.05280 |
| License | http://creativecommons.org/licenses/by-nc-nd/4.0/ |
Abstract
We formalise recursive self-training in Large Language Models (LLMs) and Generative AI as a discrete-time dynamical system and prove that, as training data become increasingly self-generated ($\alpha_t \to 0$), the system undergoes inevitably degenerative dynamics. We derive two fundamental failure modes: (1) Entropy Decay, where finite sampling effects cause a monotonic loss of distributional diversity (mode collapse), and (2) Variance Amplification, where the loss of external grounding causes the model's representation of truth to drift as a random walk, bounded only by the support diameter. We show these behaviours are not contingent on architecture but are consequences of distributional learning on finite samples. We further argue that Reinforcement Learning with imperfect verifiers suffers similar semantic collapse. To overcome these limits, we propose a path involving symbolic regression and program synthesis guided by Algorithmic Probability. The Coding Theorem Method (CTM) allows for identifying generative mechanisms rather than mere correlations, escaping the data-processing inequality that binds standard statistical learning. We conclude that while purely distributional learning leads to model collapse, hybrid neurosymbolic approaches offer a coherent framework for sustained self-improvement.
{
"annotation_id": "338fd918-ca4d-4f68-acb6-5af0b7485037",
"date_created": "2026-02-17T05:53:04.392000Z",
"date_modified": "2026-02-17T05:53:04.392000Z",
"file_hash": "2e92d156188b5a80c90f315665f7a53e3491fb40c4c4611d2272ba048595dfc3",
"private": false,
"record": {
"abstract": "We formalise recursive self-training in Large Language Models (LLMs) and Generative AI as a discrete-time dynamical system and prove that, as training data become increasingly self-generated ($\\alpha_t \\to 0$), the system undergoes inevitably degenerative dynamics. We derive two fundamental failure modes: (1) Entropy Decay, where finite sampling effects cause a monotonic loss of distributional diversity (mode collapse), and (2) Variance Amplification, where the loss of external grounding causes the model\u0027s representation of truth to drift as a random walk, bounded only by the support diameter. We show these behaviours are not contingent on architecture but are consequences of distributional learning on finite samples. We further argue that Reinforcement Learning with imperfect verifiers suffers similar semantic collapse. To overcome these limits, we propose a path involving symbolic regression and program synthesis guided by Algorithmic Probability. The Coding Theorem Method (CTM) allows for identifying generative mechanisms rather than mere correlations, escaping the data-processing inequality that binds standard statistical learning. We conclude that while purely distributional learning leads to model collapse, hybrid neurosymbolic approaches offer a coherent framework for sustained self-improvement.",
"arxiv_id": "2601.05280",
"authors": [
"Hector Zenil"
],
"categories": [
"cs.IT",
"cs.AI",
"cs.LG",
"math.IT"
],
"license": "http://creativecommons.org/licenses/by-nc-nd/4.0/",
"title": "On the Limits of Self-Improving in LLMs and Why AGI, ASI and the Singularity Are Not Near Without Symbolic Model Synthesis",
"url": "https://arxiv.org/abs/2601.05280",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "55faea68-19c1-4589-b2d6-13787b24b8a7",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}