Stationary Strong Stackelberg Equilibrium in Discounted Stochastic Games

• 成果类型: Article
• 署名作者: Lopez, Victor Bucarey; Vecchia, Eugenio Della; Jean-Marie, Alain; Ordonez, Fernando
• 署名单位: Universidad de O'Higgins; National University of Rosario; Universite Cote d'Azur; Universidad de Chile
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2022.3220512
• 发表日期: 2023
• 页码: 5271-5286
• 关键词: Optimal control; Stackelberg equilibrium; stochastic games

摘要

In this work, we study Stackelberg equilibria for discounted stochastic games. We consider two solution concepts for these games: stationary strong Stackelberg equilibrium (SSSE) and fixed point equilibrium (FPE) solutions. The SSSE solution is obtained by explicitly solving the Stackelberg equilibrium conditions, whereas the FPE can be computed efficiently using value or policy iteration algorithms. However, previous work has overlooked the relationship between these two different solution concepts. Here, we investigate the conditions for existence and equivalence of these solution concepts. Our theoretical results prove that the FPE and SSSE exist and coincide for important classes of games, including myopic follower strategy and team games. This, however, does not hold in general, and we provide numerical examples where one of SSSE or FPE does not exist, or when they both exist, they differ. Our computational results compare the solutions obtained by value iteration, policy iteration, and a mathematical programming formulations for this problem. Finally, we present a discounted stochastic Stackelberg game for a security application to illustrate the solution concepts and the efficiency of the algorithms studied.