This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Broekens, J. Articles by Verbeek, F. J. Search for Related Content Social Bookmarking                   What's this?

Affect, Anticipation, and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents

Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands, broekens{at}liacs.nl

Walter A. Kosters

Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands, kosters{at}liacs.nl

Fons J. Verbeek

Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands, fverbeek{at}liacs.nl

Emotion plays an important role in thinking. In this article we study affective control of the amount of simulated anticipatory behavior in adaptive agents using a computational model. Our approach is based on model-based reinforcement learning (RL) and inspired by the simulation hypothesis (Cotterill, 2001; Hesslow, 2002). The simulation hypothesis states that thinking is internal simulation of behavior using the same sensory-motor systems as those used for overt behavior. Here, we study the adaptiveness of an artificial agent, when action-selection bias is induced by an affect-controlled amount of simulated anticipatory behavior . To this end, we introduce an affect-controlled simulation-selection mechanism that uses the predictions of the agent's RL model to select anticipatory behaviors for simulation. Based on eXperiments with adaptive agents in two nondeterministic partially observable grid-worlds we conclude that (1) internal simulation has an adaptive benefit and (2) affective control can reduce the amount of simulation needed for this benefit. This is specifically the case if the following relation holds: positive affect decreases the amount of simulation towards simulating the best potential neXt action, while negative affect increases the amount of simulation towards simulating all potential neXt actions. In essence we use artificial affect to control mental eXploration versus eXploitations. Thus, agents "feeling positive" can think ahead in a narrow sense and free up working memory resources, while agents "feeling negative" must think ahead in a broad sense and maXimize usage of working memory. Our results are consistent with several psychological findings on the relation between affect and learning, and contribute to answering the question of when positive versus negative affect is useful during adaptation.

Key Words: affect • action selection • anticipatory simulation • simulation selection • working memory • simulated adaptive agents

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?