This Article Free Full Text (Free PDF) Free References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (7) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Stone, P. Articles by Kuhlmann, G. Search for Related Content Social Bookmarking                         What's this?

Reinforcement Learning for RoboCup Soccer Keepaway

Department of Computer Sciences, The University of Texas at Austin, pstone{at}cs.utexas.edu

Richard S. Sutton

Department of Computing Science, University of Alberta, sutton{at}cs.ualberta.ca

Gregory Kuhlmann

Department of Computer Sciences, The University of Texas at Austin, kuhlmann{at}cs.utexas.edu

RoboCup simulated soccer presents many challenges to reinforcement learning methods, including a large state space, hidden and uncertain state, multiple independent agents learning simultaneously, and long and variable delays in the effects of actions. We describe our application of episodic SMDP Sarsa() with linear tile-coding function approximation and variable to learning higher-level decisions in a keepaway subtask of RoboCup soccer. In keepaway, one team, "the keepers," tries to keep control of the ball for as long as possible despite the efforts of "the takers." The keepers learn individually when to hold the ball and when to pass to a teammate. Our agents learned policies that significantly outperform a range of benchmark policies. We demonstrate the generality of our approach by applying it to a number of task variations including different field sizes and different numbers of players on each team.

Key Words: multiagent systems • machine learning • multiagent learning • reinforcement learning • robot soccer

Adaptive Behavior, Vol. 13, No. 3, 165-188 (2005)
DOI: 10.1177/105971230501300301


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				J. L. Krichmar The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World Adaptive Behavior, December 1, 2008; 16(6): 385 - 399. [Abstract] [PDF]

				S. Whiteson, M. E. Taylor, and P. Stone Empirical Studies in Action Selection with Reinforcement Learning Adaptive Behavior, March 1, 2007; 15(1): 33 - 50. [Abstract] [PDF]