Learning With Delayed Payoffs in Population Games Using Kullback-Leibler Divergence Regularization

• 成果类型: Article
• 署名作者: Park, Shinkyu; Leonard, Naomi Ehrich
• 署名单位: King Abdullah University of Science & Technology; Princeton University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3561108
• 发表日期: 2025
• 页码: 6593-6608
• 关键词: games; Nash equilibrium; Delay effects; CONVERGENCE; vectors; roads; oscillators; Demand Response; training; stability analysis; decision making; evolutionary dynamics; game theory; Multi-agent systems; Nonlinear systems

摘要

We study a multiagent decision problem in large population games. Agents from multiple populations select strategies for repeated interactions with one another. At each stage of these interactions, agents use their decision-making model to revise their strategy selections based on payoffs determined by an underlying game. Their goal is to learn the strategies that correspond to the Nash equilibrium of the game. However, when games are subject to time delays, conventional decision-making models from the population game literature may result in oscillations in the strategy revision process or convergence to an equilibrium other than the Nash. To address this problem, we propose the Kullback-Leibler Divergence Regularized Learning (KLD-RL) model, along with an algorithm that iteratively updates the model's regularization parameter across a network of communicating agents. Using passivity-based convergence analysis techniques, we show that the KLD-RL model achieves convergence to the Nash equilibrium without oscillations, even for a class of population games that are subject to time delays. We demonstrate our main results numerically on a two-population congestion game and a two-population zero-sum game.