dorsal/arxiv
View SchemaStatistical Mechanics of Online Learning for Ensemble Teachers
| Authors | Seiji Miyoshi, Masato Okada |
|---|---|
| Categories | |
| ArXiv ID | physics/0601162 |
| URL | https://arxiv.org/abs/physics/0601162 |
| DOI | 10.1143/JPSJ.75.044002 |
Abstract
We analyze the generalization performance of a student in a model composed of linear perceptrons: a true teacher, ensemble teachers, and the student. Calculating the generalization error of the student analytically using statistical mechanics in the framework of on-line learning, it is proven that when learning rate $\eta <1$, the larger the number $K$ and the variety of the ensemble teachers are, the smaller the generalization error is. On the other hand, when $\eta >1$, the properties are completely reversed. If the variety of the ensemble teachers is rich enough, the direction cosine between the true teacher and the student becomes unity in the limit of $\eta \to 0$ and $K \to \infty$.
{
"annotation_id": "bf668103-bda9-4128-a26c-5ec38c412755",
"date_created": "2026-03-02T18:01:04.428000Z",
"date_modified": "2026-03-02T18:01:04.428000Z",
"file_hash": "1a5427b2cd1726d6e48fa5f7431f98d6e496fd4a9c774041493e35b2a814e71c",
"private": false,
"record": {
"abstract": "We analyze the generalization performance of a student in a model composed of\nlinear perceptrons: a true teacher, ensemble teachers, and the student.\nCalculating the generalization error of the student analytically using\nstatistical mechanics in the framework of on-line learning, it is proven that\nwhen learning rate $\\eta \u003c1$, the larger the number $K$ and the variety of the\nensemble teachers are, the smaller the generalization error is. On the other\nhand, when $\\eta \u003e1$, the properties are completely reversed. If the variety of\nthe ensemble teachers is rich enough, the direction cosine between the true\nteacher and the student becomes unity in the limit of $\\eta \\to 0$ and $K \\to\n\\infty$.",
"arxiv_id": "physics/0601162",
"authors": [
"Seiji Miyoshi",
"Masato Okada"
],
"categories": [
"physics.soc-ph",
"cond-mat.dis-nn"
],
"doi": "10.1143/JPSJ.75.044002",
"title": "Statistical Mechanics of Online Learning for Ensemble Teachers",
"url": "https://arxiv.org/abs/physics/0601162"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "b91a9c99-185a-4c56-9b20-6a9decd68d23",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}