This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Jung, T. Articles by Uthmann, T. Search for Related Content Social Bookmarking                         What's this?

Evolution and Learning: Evolving Sensors in a Simple MDP Environment

Institut für Informatik, Gutenberg-Universität Mainz, tjung{at}informatik.uni-mainz.de

Peter Dauscher

Institut für Informatik, Gutenberg-Universität Mainz, dauscher{at}informatik.uni-mainz.de

Thomas Uthmann

Institut für Informatik, Gutenberg-Universität Mainz, uthmann{at}informatik.uni-mainz.de

Natural intelligence and autonomous agents face difficulties when acting in information-dense environments. Assailed by a multitude of stimuli they have to make sense of the inflow of information, filtering and processing what is necessary, but discarding that which is unimportant. This paper aims at investigating the interactions between evolution of the sensorial channel extracting the information from the environment and the simultaneous individual adaptation of agent-control. Our particular goal is to study the influence of learning on the evolution of sensors, with learning duration being the tunable parameter. A genetic algorithm governs the evolution of sensors appropriate for the agent solving a simple grid world task. The performance of the agent is taken as fitness; ‘sensors’ are conceived as a map from environmental states to agent observations, and individual adaptation is modeled by Q-learning. Our experimental results show that due to the principles of cognitive economy learning and varying the degree thereof actually transforms the fitness landscape. In particular we identify a trade-off between learning speed (load) and sensor accuracy (error). These results are further reinforced by theoretical analysis: we derive an analytical measure for the quality of sensors based on the mutual entropy between the system of states and the selection of an optimal action, a concept recently proposed by Polani, Martinetz, and Kim.

Key Words: sensor evolution • adaptive • state generalization • cognitive economy • relevant information • reinforcement learning

Adaptive Behavior, Vol. 11, No. 3, 159-177 (2003)
DOI: 10.1177/1059712303113002


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?