This Article Free Full Text (Free PDF) Free References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Banerjee, B. Articles by Peng, J. Search for Related Content Social Bookmarking                   What's this?

Reactivity and Safe Learning in Multi-Agent Systems

Department of Electrical Engineering and Computer Science, Tulane University, New Orleans, LA 70118, USA, banerjee{at}eecs.tulane.edu

Jing Peng

Department of Electrical Engineering and Computer Science, Tulane University, New Orleans, LA 70118, USA

Multi-agent reinforcement learning (MRL) is a growing area of research. What makes it particularly challenging is that multiple learners render each other's environments non-stationary. In addition to adapting their behaviors to other learning agents, online learners must also provide assurances about their online performance in order to promote user trust of adaptive agent systems deployed in real world applications. In this article, instead of developing new algorithms with such assurances, we study the question of safety in online performance of some existing MRL algorithms. We identify the key notion of reactivity of a learner by analyzing how an algorithm (PHC-Exploiter), designed to exploit some simpler opponents, can itself be exploited by them. We quantify and analyze this concept of reac tivity in the context of these algorithms to explain their experimental behaviors. We argue that no learner can be designed that can deliberately avoid exploitation. We also show that any attempt to opti mize reactivity must take into account a tradeoff with sensitivity to noise, and devise an adaptive method (based on environmental feedback) designed to maximize the learner's safety and minimize its sensitivity to noise.

Key Words: multi-agent systems • reinforcement learning • game theory

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?