dorsal/arxiv
View SchemaInnovation Capacity of Dynamical Learning Systems
| Authors | Anthony M. Polloreno |
|---|---|
| Categories | |
| ArXiv ID | 2601.07257vv1 |
| URL | https://arxiv.org/abs/2601.07257 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
In noisy physical reservoirs, the classical information-processing capacity $C_{\mathrm{ip}}$ quantifies how well a linear readout can realize tasks measurable from the input history, yet $C_{\mathrm{ip}}$ can be far smaller than the observed rank of the readout covariance. We explain this ``missing capacity'' by introducing the innovation capacity $C_{\mathrm{i}}$, the total capacity allocated to readout components orthogonal to the input filtration (Doob innovations, including input-noise mixing). Using a basis-free Hilbert-space formulation of the predictable/innovation decomposition, we prove the conservation law $C_{\mathrm{ip}}+C_{\mathrm{i}}=\mathrm{rank}(\Sigma_{XX})\le d$, so predictable and innovation capacities exactly partition the rank of the observable readout dimension covariance $\Sigma_{XX}\in \mathbb{R}^{\rm d\times d}$. In linear-Gaussian Johnson-Nyquist regimes, $\Sigma_{XX}(T)=S+T N_0$, the split becomes a generalized-eigenvalue shrinkage rule and gives an explicit monotone tradeoff between temperature and predictable capacity. Geometrically, in whitened coordinates the predictable and innovation components correspond to complementary covariance ellipsoids, making $C_{\mathrm{i}}$ a trace-controlled innovation budget. A large $C_{\mathrm{i}}$ forces a high-dimensional innovation subspace with a variance floor and under mild mixing and anti-concentration assumptions this yields extensive innovation-block differential entropy and exponentially many distinguishable histories. Finally, we give an information-theoretic lower bound showing that learning the induced innovation-block law in total variation requires a number of samples that scales with the effective innovation dimension, supporting the generative utility of noisy physical reservoirs.
{
"annotation_id": "88ed83dd-a5db-4ead-bdf6-53fa75d51cde",
"date_created": "2026-02-17T05:53:12.009000Z",
"date_modified": "2026-02-17T05:53:12.009000Z",
"file_hash": "f0cd8dd07280e8d09acacb0e8fd6bcc033a5014d17c966b3a119d7139ae9d4e7",
"private": false,
"record": {
"abstract": "In noisy physical reservoirs, the classical information-processing capacity $C_{\\mathrm{ip}}$ quantifies how well a linear readout can realize tasks measurable from the input history, yet $C_{\\mathrm{ip}}$ can be far smaller than the observed rank of the readout covariance. We explain this ``missing capacity\u0027\u0027 by introducing the innovation capacity $C_{\\mathrm{i}}$, the total capacity allocated to readout components orthogonal to the input filtration (Doob innovations, including input-noise mixing). Using a basis-free Hilbert-space formulation of the predictable/innovation decomposition, we prove the conservation law $C_{\\mathrm{ip}}+C_{\\mathrm{i}}=\\mathrm{rank}(\\Sigma_{XX})\\le d$, so predictable and innovation capacities exactly partition the rank of the observable readout dimension covariance $\\Sigma_{XX}\\in \\mathbb{R}^{\\rm d\\times d}$. In linear-Gaussian Johnson-Nyquist regimes, $\\Sigma_{XX}(T)=S+T N_0$, the split becomes a generalized-eigenvalue shrinkage rule and gives an explicit monotone tradeoff between temperature and predictable capacity. Geometrically, in whitened coordinates the predictable and innovation components correspond to complementary covariance ellipsoids, making $C_{\\mathrm{i}}$ a trace-controlled innovation budget. A large $C_{\\mathrm{i}}$ forces a high-dimensional innovation subspace with a variance floor and under mild mixing and anti-concentration assumptions this yields extensive innovation-block differential entropy and exponentially many distinguishable histories. Finally, we give an information-theoretic lower bound showing that learning the induced innovation-block law in total variation requires a number of samples that scales with the effective innovation dimension, supporting the generative utility of noisy physical reservoirs.",
"arxiv_id": "2601.07257",
"authors": [
"Anthony M. Polloreno"
],
"categories": [
"cs.LG",
"cs.IT",
"math.IT"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Innovation Capacity of Dynamical Learning Systems",
"url": "https://arxiv.org/abs/2601.07257",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "6178ceb3-3186-4755-83f7-c693a5a5b010",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}