dorsal/arxiv
View SchemaEntropy Estimates from Insufficient Samplings
| Authors | P. Grassberger |
|---|---|
| Categories | |
| ArXiv ID | physics/0307138 |
| URL | https://arxiv.org/abs/physics/0307138 |
Abstract
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1 points in each box) they have exponentially small biases. In the low sampling regime the errors increase but are still much smaller than for most other estimators. One advantage is that our main estimators are given analytically, with explicitly known analytical formulas for the biases.
{
"annotation_id": "40ab8834-6b04-443e-a056-0f6d06405d34",
"date_created": "2026-03-02T18:00:46.926000Z",
"date_modified": "2026-03-02T18:00:46.926000Z",
"file_hash": "bb29da99d0e792ae74e2d3f709ee2a97727d320cef2bffb18d2587b1eccee6a0",
"private": false,
"record": {
"abstract": "We present a detailed derivation of some estimators of Shannon entropy for\ndiscrete distributions. They hold for finite samples of N points distributed\ninto M \"boxes\", with N and M -\u003e oo, but N/M \u003c oo. In the high sampling regime\n(\u003c\u003c 1 points in each box) they have exponentially small biases. In the low\nsampling regime the errors increase but are still much smaller than for most\nother estimators. One advantage is that our main estimators are given\nanalytically, with explicitly known analytical formulas for the biases.",
"arxiv_id": "physics/0307138",
"authors": [
"P. Grassberger"
],
"categories": [
"physics.data-an",
"physics.comp-ph"
],
"title": "Entropy Estimates from Insufficient Samplings",
"url": "https://arxiv.org/abs/physics/0307138"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "45d602af-54b9-4f49-86f7-0d32265eb7a6",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}