dorsal/arxiv
View SchemaEntropic Priors for Discrete Probabilistic Networks and for Mixtures of Gaussians Models
| Authors | Carlos C. Rodriguez |
|---|---|
| Categories | |
| ArXiv ID | physics/0201016 |
| URL | https://arxiv.org/abs/physics/0201016 |
| DOI | 10.1063/1.1477063 |
Abstract
The ongoing unprecedented exponential explosion of available computing power, has radically transformed the methods of statistical inference. What used to be a small minority of statisticians advocating for the use of priors and a strict adherence to bayes theorem, it is now becoming the norm across disciplines. The evolutionary direction is now clear. The trend is towards more realistic, flexible and complex likelihoods characterized by an ever increasing number of parameters. This makes the old question of: What should the prior be? to acquire a new central importance in the modern bayesian theory of inference. Entropic priors provide one answer to the problem of prior selection. The general definition of an entropic prior has existed since 1988, but it was not until 1998 that it was found that they provide a new notion of complete ignorance. This paper re-introduces the family of entropic priors as minimizers of mutual information between the data and the parameters, as in [rodriguez98b], but with a small change and a correction. The general formalism is then applied to two large classes of models: Discrete probabilistic networks and univariate finite mixtures of gaussians. It is also shown how to perform inference by efficiently sampling the corresponding posterior distributions.
{
"annotation_id": "988eaffc-5164-4132-8ee1-0995c3dbe633",
"date_created": "2026-03-02T18:00:38.970000Z",
"date_modified": "2026-03-02T18:00:38.970000Z",
"file_hash": "96fdb729646280dfa9593c2f2ecaa52a58876f447885ce4347581ec9a6756f75",
"private": false,
"record": {
"abstract": "The ongoing unprecedented exponential explosion of available computing power,\nhas radically transformed the methods of statistical inference. What used to be\na small minority of statisticians advocating for the use of priors and a strict\nadherence to bayes theorem, it is now becoming the norm across disciplines. The\nevolutionary direction is now clear. The trend is towards more realistic,\nflexible and complex likelihoods characterized by an ever increasing number of\nparameters. This makes the old question of: What should the prior be? to\nacquire a new central importance in the modern bayesian theory of inference.\nEntropic priors provide one answer to the problem of prior selection. The\ngeneral definition of an entropic prior has existed since 1988, but it was not\nuntil 1998 that it was found that they provide a new notion of complete\nignorance. This paper re-introduces the family of entropic priors as minimizers\nof mutual information between the data and the parameters, as in\n[rodriguez98b], but with a small change and a correction. The general formalism\nis then applied to two large classes of models: Discrete probabilistic networks\nand univariate finite mixtures of gaussians. It is also shown how to perform\ninference by efficiently sampling the corresponding posterior distributions.",
"arxiv_id": "physics/0201016",
"authors": [
"Carlos C. Rodriguez"
],
"categories": [
"physics.data-an"
],
"doi": "10.1063/1.1477063",
"title": "Entropic Priors for Discrete Probabilistic Networks and for Mixtures of Gaussians Models",
"url": "https://arxiv.org/abs/physics/0201016"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "8fea3dcf-b61c-43f2-ae49-34bb8ab2471e",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}