dorsal/arxiv
View SchemaMaximum Entropy Multivariate Density Estimation: An exact goodness-of-fit approach
| Authors | Sabbir Rahman, Mahbub Majumdar |
|---|---|
| Categories | |
| ArXiv ID | physics/0406023 |
| URL | https://arxiv.org/abs/physics/0406023 |
Abstract
We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the problem considers contributions only from the smoothness of the estimated distribution (as measured by its entropy) and the loss functional associated with its goodness-of-fit to the sample data, and in particular does not make use of any additional constraints that cannot be justified from the sample data alone. By mapping the general multivariate problem to a tractable univariate one, we are able to write down exact expressions for the goodness-of-fit of an arbitrary multivariate distribution to any given set of samples using both the traditional likelihood-based approach and a rigorous information-theoretic approach, thus solving a long-standing problem. As a corollary we also give an exact solution to the `forward problem' of determining the expected distributions of samples taken from a population with known probability distribution.
{
"annotation_id": "27a18151-1b34-481f-90a6-20cf4a36dfb3",
"date_created": "2026-03-02T18:00:50.494000Z",
"date_modified": "2026-03-02T18:00:50.494000Z",
"file_hash": "5b3412ef01058570f87f35a0f5b0be0c91c0971bb4e3fe12935f1d4abb71f0f2",
"private": false,
"record": {
"abstract": "We consider the problem of estimating the population probability distribution\ngiven a finite set of multivariate samples, using the maximum entropy approach.\nIn strict keeping with Jaynes\u0027 original definition, our precise formulation of\nthe problem considers contributions only from the smoothness of the estimated\ndistribution (as measured by its entropy) and the loss functional associated\nwith its goodness-of-fit to the sample data, and in particular does not make\nuse of any additional constraints that cannot be justified from the sample data\nalone. By mapping the general multivariate problem to a tractable univariate\none, we are able to write down exact expressions for the goodness-of-fit of an\narbitrary multivariate distribution to any given set of samples using both the\ntraditional likelihood-based approach and a rigorous information-theoretic\napproach, thus solving a long-standing problem. As a corollary we also give an\nexact solution to the `forward problem\u0027 of determining the expected\ndistributions of samples taken from a population with known probability\ndistribution.",
"arxiv_id": "physics/0406023",
"authors": [
"Sabbir Rahman",
"Mahbub Majumdar"
],
"categories": [
"physics.data-an",
"cs.IT",
"math.IT",
"math.ST",
"stat.TH"
],
"title": "Maximum Entropy Multivariate Density Estimation: An exact goodness-of-fit approach",
"url": "https://arxiv.org/abs/physics/0406023"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "e645057a-2193-432f-b1d6-e258c898b6a5",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}