Markov α-Potential Games

• 成果类型: Article
• 署名作者: Guo, Xin; Li, Xinyu; Maheshwari, Chinmay; Sastry, Shankar; Wu, Manxi
• 署名单位: University of California System; University of California Berkeley; Johns Hopkins University; University of California System; University of California Berkeley; University of California System; University of California Berkeley
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3589416
• 发表日期: 2026
• 关键词: REINFORCEMENT

摘要

In this article, we propose a new framework of Markov alpha-potential games to study Markov games. We show that any Markov game with finite state and finite action is a Markov alpha-potential game and establish the existence of an associated alpha-potential function. Any optimizer of an alpha-potential function is shown to be an alpha-stationary Nash equilibrium. We study two important classes of practically significant Markov games, Markov congestion games and the perturbed Markov team games, via the framework of Markov alpha-potential games, with explicit characterization of an upper bound for alpha and its relation to game parameters. In addition, we provide a semi-infinite linear programming-based formulation to obtain an upper bound for alpha for any Markov game. Furthermore, we study two equilibrium approximation algorithms, namely, the projected gradient-ascent algorithm and the sequential maximum improvement algorithm, along with their Nash regret analysis, and corroborate the results with numerical experiments.