Attainability of boundary points under reinforcement learning

• 成果类型: Article
• 署名作者: Hopkins, E; Posch, M
• 署名单位: University of Edinburgh; Medical University of Vienna
• 刊物名称: GAMES AND ECONOMIC BEHAVIOR
• ISSN/ISSBN: 0899-8256
• DOI: 10.1016/j.geb.2004.08.002
• 发表日期: 2005
• 页码: 110-125
• 关键词: Learning in games; Reinforcement Learning; stochastic approximation; Replicator dynamics

摘要

This paper investigates the properties of the most common form of reinforcement learning (the basic model of Erev and Roth) [Amer. Econ. Rev. 88 (1998) 848-881]. Stochastic approximation theory has been used to analyse the local stability of fixed points under this learning process. However, as we show, when such points are on the boundary of the state space, for example, pure strategy equilibria, standard results from the theory of stochastic approximation do not apply. We offer what we believe to be the correct treatment of boundary points, and provide a new and more general result: this model of learning converges with zero probability to fixed points which are unstable under the Maynard Smith or adjusted version of the evolutionary replicator dynamics. For two player games these are the fixed points that are linearly unstable under the standard replicator dynamics. (c) 2004 Elsevier Inc. All rights reserved.