This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (3) Citing Articles via Google Scholar Google Scholar Articles by Khamassi, M. Articles by Guillot, A. Search for Related Content Social Bookmarking                   What's this?

Actor–Critic Models of Reinforcement Learning in the Basal Ganglia: From Natural to Artificial Rats

AnimatLab, LIP6, Paris, France; LPPA, CNRS–Collège de France, Paris, France

Loïc Lachèze

AnimatLab, LIP6, Paris, France

Benoît Girard

AnimatLab, LIP6, Paris, France; LPPA, CNRS–Collège de France, Paris, France

Alain Berthoz

LPPA, CNRS–Collège de France, Paris, France

Agnès Guillot

AnimatLab, LIP6, Paris, France

Since 1995, numerous Actor–Critic architectures for reinforcement learning have been proposed as models of dopamine-like reinforcement learning mechanisms in the rat’s basal ganglia. However, these models were usually tested in different tasks, and it is then difficult to compare their efficiency for an autonomous animat. We present here the comparison of four architectures in an animat as it per forms the same reward-seeking task. This will illustrate the consequences of different hypotheses about the management of different Actor sub-modules and Critic units, and their more or less autono mously determined coordination. We show that the classical method of coordination of modules by mixture of experts, depending on each module’s performance, did not allow solving our task. Then we address the question of which principle should be applied efficiently to combine these units. Improve ments for Critic modeling and accuracy of Actor–Critic models for a natural task are finally discussed in the perspective of our Psikharpax project—an artificial rat having to survive autonomously in unpre dictable environments.

Key Words: animat approach • TD learning • Actor–Critic model • S–R task • taxon navigation

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				M. Witkowski An Action-Selection Calculus Adaptive Behavior, March 1, 2007; 15(1): 73 - 97. [Abstract] [PDF]