dorsal/arxiv
View SchemaDynamical transitions in the evolution of learning algorithms by selection
| Authors | Juan Pablo Neirotti, Nestor Caticha |
|---|---|
| Categories | |
| ArXiv ID | physics/0209048 |
| URL | https://arxiv.org/abs/physics/0209048 |
| DOI | 10.1103/PhysRevE.67.041912 |
Abstract
We study the evolution of artificial learning systems by means of selection. Genetic programming is used to generate a sequence of populations of algorithms which can be used by neural networks for supervised learning of a rule that generates examples. In opposition to concentrating on final results, which would be the natural aim while designing good learning algorithms, we study the evolution process and pay particular attention to the temporal order of appearance of functional structures responsible for the improvements in the learning process, as measured by the generalization capabilities of the resulting algorithms. The effect of such appearances can be described as dynamical phase transitions. The concepts of phenotypic and genotypic entropies, which serve to describe the distribution of fitness in the population and the distribution of symbols respectively, are used to monitor the dynamics. In different runs the phase transitions might be present or not, with the system finding out good solutions, or staying in poor regions of algorithm space. Whenever phase transitions occur, the sequence of appearances are the same. We identify combinations of variables and operators which are useful in measuring experience or performance in rule extraction and can thus implement useful annealing of the learning schedule.
{
"annotation_id": "e9517a16-c8ae-447e-b633-74509bf891d4",
"date_created": "2026-03-02T18:00:39.812000Z",
"date_modified": "2026-03-02T18:00:39.812000Z",
"file_hash": "343a8bf090f26691831d5e342a68f0f430c2c885fb7eed9f1100db92a1308a6e",
"private": false,
"record": {
"abstract": "We study the evolution of artificial learning systems by means of selection.\nGenetic programming is used to generate a sequence of populations of algorithms\nwhich can be used by neural networks for supervised learning of a rule that\ngenerates examples. In opposition to concentrating on final results, which\nwould be the natural aim while designing good learning algorithms, we study the\nevolution process and pay particular attention to the temporal order of\nappearance of functional structures responsible for the improvements in the\nlearning process, as measured by the generalization capabilities of the\nresulting algorithms. The effect of such appearances can be described as\ndynamical phase transitions. The concepts of phenotypic and genotypic\nentropies, which serve to describe the distribution of fitness in the\npopulation and the distribution of symbols respectively, are used to monitor\nthe dynamics. In different runs the phase transitions might be present or not,\nwith the system finding out good solutions, or staying in poor regions of\nalgorithm space. Whenever phase transitions occur, the sequence of appearances\nare the same. We identify combinations of variables and operators which are\nuseful in measuring experience or performance in rule extraction and can thus\nimplement useful annealing of the learning schedule.",
"arxiv_id": "physics/0209048",
"authors": [
"Juan Pablo Neirotti",
"Nestor Caticha"
],
"categories": [
"physics.bio-ph",
"cond-mat.dis-nn"
],
"doi": "10.1103/PhysRevE.67.041912",
"title": "Dynamical transitions in the evolution of learning algorithms by selection",
"url": "https://arxiv.org/abs/physics/0209048"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "fc948527-4c4c-46b2-9a04-19b52eb69d2a",
"id": "arXiv Dataset IDs",
"type": "Model",
"variant": "snapshot-2026-03-01",
"version": "0.1.0"
},
"user_id": 1000002
}