This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Alexander, W. H. Search for Related Content Social Bookmarking                         What's this?

Shifting Attention Using a Temporal Difference Prediction Error and High-Dimensional Input

Initial Research Project, Okinawa Institute for Science and Technology, Okinawa, Japan, wialexan{at}oist.jp

Research on reinforcement learning has increasingly focused on the role of neuromodulatory systems implicated in associative learning. Formulations of temporal difference (TD) learning have gained a great deal of attention due to the similarity of the TD prediction error and the observed activity of dopamine neurons in the primate midbrain. Recent work has attempted to integrate additional neuromodulatory systems such as noradrenaline and acetylcholine in a TD framework. Additional work has been done to remedy representational issues arising from TD variants that result in incorrect predictions of dopamine activity, as well as to incorporate the TD error signal in models of categorization. In this paper, an actor—critic model incorporating aspects of TD learning and psychological models of attention is described. The development of the model and the behavior of an autonomous agent in a simulated environment are examined and compared with a variant of TD learning lacking an attentional component. The agent learns to behave adaptively due to the shifting of attention to relevant aspects of a high-dimensional input. In contrast, the TD model exhibits perseverative behavior and comparatively slow learning in the same context. It is suggested that real-time models of attention may provide insight into neuromodulatory systems implicated in attention and representational learning.

Key Words: attention • temporal difference learning • reinforcement learning • representation • adaptive behavior

Adaptive Behavior, Vol. 15, No. 2, 121-133 (2007)
DOI: 10.1177/1059712307078663


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?