dorsal/arxiv
View SchemaPenalized maximum likelihood for multivariate Gaussian mixture
| Authors | Hichem Snoussi, Ali Mohammad-Djafari |
|---|---|
| Categories | |
| ArXiv ID | physics/0111007 |
| URL | https://arxiv.org/abs/physics/0111007 |
| DOI | 10.1063/1.1477037 |
Abstract
In this paper, we first consider the parameter estimation of a multivariate random process distribution using multivariate Gaussian mixture law. The labels of the mixture are allowed to have a general probability law which gives the possibility to modelize a temporal structure of the process under study. We generalize the case of univariate Gaussian mixture in [Ridolfi99] to show that the likelihood is unbounded and goes to infinity when one of the covariance matrices approaches the boundary of singularity of the non negative definite matrices set. We characterize the parameter set of these singularities. As a solution to this degeneracy problem, we show that the penalization of the likelihood by an Inverse Wishart prior on covariance matrices results to a penalized or maximum a posteriori criterion which is bounded. Then, the existence of positive definite matrices optimizing this criterion can be guaranteed. We also show that with a modified EM procedure or with a Bayesian sampling scheme, we can constrain covariance matrices to belong to a particular subclass of covariance matrices. Finally, we study degeneracies in the source separation problem where the characterization of parameter singularity set is more complex. We show, however, that Inverse Wishart prior on covariance matrices eliminates the degeneracies in this case too.
{
"annotation_id": "81b27662-5db3-49a8-a3a7-514fc005a2f5",
"date_created": "2026-03-02T18:00:36.396000Z",
"date_modified": "2026-03-02T18:00:36.396000Z",
"file_hash": "776fd19b4ae8735f24d3c73ebf0fa01cbbd0a634745cf225754cd08ed35e15d7",
"private": false,
"record": {
"abstract": "In this paper, we first consider the parameter estimation of a multivariate\nrandom process distribution using multivariate Gaussian mixture law. The labels\nof the mixture are allowed to have a general probability law which gives the\npossibility to modelize a temporal structure of the process under study. We\ngeneralize the case of univariate Gaussian mixture in [Ridolfi99] to show that\nthe likelihood is unbounded and goes to infinity when one of the covariance\nmatrices approaches the boundary of singularity of the non negative definite\nmatrices set. We characterize the parameter set of these singularities. As a\nsolution to this degeneracy problem, we show that the penalization of the\nlikelihood by an Inverse Wishart prior on covariance matrices results to a\npenalized or maximum a posteriori criterion which is bounded. Then, the\nexistence of positive definite matrices optimizing this criterion can be\nguaranteed. We also show that with a modified EM procedure or with a Bayesian\nsampling scheme, we can constrain covariance matrices to belong to a particular\nsubclass of covariance matrices. Finally, we study degeneracies in the source\nseparation problem where the characterization of parameter singularity set is\nmore complex. We show, however, that Inverse Wishart prior on covariance\nmatrices eliminates the degeneracies in this case too.",
"arxiv_id": "physics/0111007",
"authors": [
"Hichem Snoussi",
"Ali Mohammad-Djafari"
],
"categories": [
"physics.data-an"
],
"doi": "10.1063/1.1477037",
"title": "Penalized maximum likelihood for multivariate Gaussian mixture",
"url": "https://arxiv.org/abs/physics/0111007"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "762bc6c4-10b8-47eb-8cd3-6d853bc79283",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}