dorsal/arxiv
View SchemaMaximally Informative Statistics
| Authors | David R. Wolf, Edward I. George |
|---|---|
| Categories | |
| ArXiv ID | physics/0010039 |
| URL | https://arxiv.org/abs/physics/0010039 |
| Journal | Monograph on Bayesian Methods in the Sciences, Rev. R. Acad. Sci. Exacta. Fisica. Nat. Vol. 93, No. 3, pp. 381--386, 1999 |
Abstract
In this paper we propose a Bayesian, information theoretic approach to dimensionality reduction. The approach is formulated as a variational principle on mutual information, and seamlessly addresses the notions of sufficiency, relevance, and representation. Maximally informative statistics are shown to minimize a Kullback-Leibler distance between posterior distributions. Illustrating the approach, we derive the maximally informative one dimensional statistic for a random sample from the Cauchy distribution.
{
"annotation_id": "ee148697-0caf-407e-bee8-0a142034b97a",
"date_created": "2026-03-02T18:00:32.237000Z",
"date_modified": "2026-03-02T18:00:32.237000Z",
"file_hash": "7d51c1002e57e3ea2d9ece70c38596ca15b305f8daf847d99327ea0784e27d9e",
"private": false,
"record": {
"abstract": "In this paper we propose a Bayesian, information theoretic approach to\ndimensionality reduction. The approach is formulated as a variational principle\non mutual information, and seamlessly addresses the notions of sufficiency,\nrelevance, and representation. Maximally informative statistics are shown to\nminimize a Kullback-Leibler distance between posterior distributions.\nIllustrating the approach, we derive the maximally informative one dimensional\nstatistic for a random sample from the Cauchy distribution.",
"arxiv_id": "physics/0010039",
"authors": [
"David R. Wolf",
"Edward I. George"
],
"categories": [
"physics.data-an"
],
"journal_ref": "Monograph on Bayesian Methods in the Sciences, Rev. R. Acad. Sci.\n Exacta. Fisica. Nat. Vol. 93, No. 3, pp. 381--386, 1999",
"title": "Maximally Informative Statistics",
"url": "https://arxiv.org/abs/physics/0010039"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "8685aa1f-e964-43f7-873b-63b4343e2a65",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}