dorsal/arxiv
View SchemaMaximum Entropy method with non-linear moment constraints: challenges
| Authors | M. Grendar, Jr., M. Grendar |
|---|---|
| Categories | |
| ArXiv ID | physics/0308006 |
| URL | https://arxiv.org/abs/physics/0308006 |
| Journal | In: Bayesian inference and Maximum Entropy methods in Science and Engineering, G. Erickson and Y. Zhai (eds.), AIP (Melville), 97-109, 2004 |
Abstract
Traditionally, the Method of (Shannon-Kullback's) Relative Entropy Maximization (REM) is considered with linear moment constraints. In this work, the method is studied under frequency moment constraints which are non-linear in probabilities. The constraints challenge some justifications of REM since a) axiomatic systems are developed for classical linear moment constraints, b) the feasible set of distributions which is defined by frequency moment constraints admits several entropy maximizing distributions (I-projections), hence probabilistic justification of REM via Conditioned Weak Law of Large Numbers cannot be invoked. However, REM is not left completely unjustified in this setting, since Entropy Concentration Theorem and Maximum Probability Theorem can be applied. Maximum Renyi-Tsallis' entropy method (maxTent) enters this work because of non-linearity of X-frequency moment constraints which are used in Non-extensive Thermodynamics. It is shown here that under X-frequency moment constraints maxTent distribution can be unique and different than the I-projection. This implies that maxTent does not choose the most probable distribution and that the maxTent distribution is asymptotically conditionally improbable. What are adherents of maxTent accomplishing when they maximize Renyi's or Tsallis' entropy?
{
"annotation_id": "4d1d7666-8068-42dc-920f-f2ec4f23c5fd",
"date_created": "2026-03-02T18:00:46.929000Z",
"date_modified": "2026-03-02T18:00:46.929000Z",
"file_hash": "6fdf688ea4687506c918c8c7ca91c587981b77e20c8c94fbf2174b9271ce6c6e",
"private": false,
"record": {
"abstract": "Traditionally, the Method of (Shannon-Kullback\u0027s) Relative Entropy\nMaximization (REM) is considered with linear moment constraints. In this work,\nthe method is studied under frequency moment constraints which are non-linear\nin probabilities. The constraints challenge some justifications of REM since a)\naxiomatic systems are developed for classical linear moment constraints, b) the\nfeasible set of distributions which is defined by frequency moment constraints\nadmits several entropy maximizing distributions (I-projections), hence\nprobabilistic justification of REM via Conditioned Weak Law of Large Numbers\ncannot be invoked. However, REM is not left completely unjustified in this\nsetting, since Entropy Concentration Theorem and Maximum Probability Theorem\ncan be applied.\n Maximum Renyi-Tsallis\u0027 entropy method (maxTent) enters this work because of\nnon-linearity of X-frequency moment constraints which are used in Non-extensive\nThermodynamics. It is shown here that under X-frequency moment constraints\nmaxTent distribution can be unique and different than the I-projection. This\nimplies that maxTent does not choose the most probable distribution and that\nthe maxTent distribution is asymptotically conditionally improbable. What are\nadherents of maxTent accomplishing when they maximize Renyi\u0027s or Tsallis\u0027\nentropy?",
"arxiv_id": "physics/0308006",
"authors": [
"M. Grendar, Jr.",
"M. Grendar"
],
"categories": [
"physics.data-an"
],
"journal_ref": "In: Bayesian inference and Maximum Entropy methods in Science and\n Engineering, G. Erickson and Y. Zhai (eds.), AIP (Melville), 97-109, 2004",
"title": "Maximum Entropy method with non-linear moment constraints: challenges",
"url": "https://arxiv.org/abs/physics/0308006"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "8337da58-d43e-4340-88f2-b8cb52f6bf97",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}