dorsal/arxiv
View SchemaLeast Dependent Component Analysis Based on Mutual Information
| Authors | Harald Stögbauer, Alexander Kraskov, Sergey A. Astakhov, Peter Grassberger |
|---|---|
| Categories | |
| ArXiv ID | physics/0405044 |
| URL | https://arxiv.org/abs/physics/0405044 |
| DOI | 10.1103/PhysRevE.70.066123 |
| Journal | Phys. Rev. E 70, 066123 (2004) |
Abstract
We propose to use precise estimators of mutual information (MI) to find least dependent components in a linearly mixed signal. On the one hand this seems to lead to better blind source separation than with any other presently available algorithm. On the other hand it has the advantage, compared to other implementations of `independent' component analysis (ICA) some of which are based on crude approximations for MI, that the numerical values of the MI can be used for: (i) estimating residual dependencies between the output components; (ii) estimating the reliability of the output, by comparing the pairwise MIs with those of re-mixed components; (iii) clustering the output according to the residual interdependencies. For the MI estimator we use a recently proposed k-nearest neighbor based algorithm. For time sequences we combine this with delay embedding, in order to take into account non-trivial time correlations. After several tests with artificial data, we apply the resulting MILCA (Mutual Information based Least dependent Component Analysis) algorithm to a real-world dataset, the ECG of a pregnant woman. The software implementation of the MILCA algorithm is freely available at http://www.fz-juelich.de/nic/cs/software
{
"annotation_id": "f321100b-bdb0-4315-86b1-c51994adb557",
"date_created": "2026-03-02T18:00:50.253000Z",
"date_modified": "2026-03-02T18:00:50.253000Z",
"file_hash": "df099e61b1ec78de1f1ca235df1097c9be21816cd24a4b8e99eab76299cb901e",
"private": false,
"record": {
"abstract": "We propose to use precise estimators of mutual information (MI) to find least\ndependent components in a linearly mixed signal. On the one hand this seems to\nlead to better blind source separation than with any other presently available\nalgorithm. On the other hand it has the advantage, compared to other\nimplementations of `independent\u0027 component analysis (ICA) some of which are\nbased on crude approximations for MI, that the numerical values of the MI can\nbe used for:\n (i) estimating residual dependencies between the output components;\n (ii) estimating the reliability of the output, by comparing the pairwise MIs\nwith those of re-mixed components;\n (iii) clustering the output according to the residual interdependencies.\n For the MI estimator we use a recently proposed k-nearest neighbor based\nalgorithm. For time sequences we combine this with delay embedding, in order to\ntake into account non-trivial time correlations. After several tests with\nartificial data, we apply the resulting MILCA (Mutual Information based Least\ndependent Component Analysis) algorithm to a real-world dataset, the ECG of a\npregnant woman.\n The software implementation of the MILCA algorithm is freely available at\nhttp://www.fz-juelich.de/nic/cs/software",
"arxiv_id": "physics/0405044",
"authors": [
"Harald St\u00f6gbauer",
"Alexander Kraskov",
"Sergey A. Astakhov",
"Peter Grassberger"
],
"categories": [
"physics.comp-ph",
"cs.IT",
"math.IT",
"physics.data-an",
"q-bio.QM"
],
"doi": "10.1103/PhysRevE.70.066123",
"journal_ref": "Phys. Rev. E 70, 066123 (2004)",
"title": "Least Dependent Component Analysis Based on Mutual Information",
"url": "https://arxiv.org/abs/physics/0405044"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "f9d610dc-9c4d-46c7-b987-611fa813d1e4",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}