Deep Reinforcement Learning for Equilibrium Computation in Multistage Auctions and Contests

• 成果类型: Article; Early Access
• 署名作者: Pieroth, Fabian R.; Kohring, Nils; Bichler, Martin
• 署名单位: Technical University of Munich
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2024.06771
• 发表日期: 2025
• 关键词: deep reinforcement learning; Nash equilibrium; auctions; contests

摘要

We compute equilibrium strategies in multistage games with continuous signal and action spaces as they are widely used in the management sciences and economics. Examples include sequential sales via auctions, multistage elimination contests, and Stackelberg competitions. In sequential auctions, analysts performing equilibrium analysis are required to derive not just single bids but bid functions for all possible signals or values that a bidder might have in multiple stages. Because of the continuity of the signal and action spaces, these bid functions come from an infinite dimensional space. Although such models are fundamental to game theory and its applications, equilibrium strategies are rarely known. The resulting system of nonlinear differential equations is considered intractable for all but elementary models. This has been limiting progress in game theory and is a barrier to its adoption in the field. We show that deep reinforcement learning and selfplay can learn equilibrium bidding strategies for various multistage games. Verifying an equilibrium in such games is challenging because of the continuous signal and action spaces. We introduce a verification algorithm and prove that the error of this verifier decreases when considering Lipschitz continuous strategies with increasing levels of discretization and sample sizes. Leveraging the novel verification algorithm, we find equilibrium in models that have not yet been explored analytically and new asymmetric equilibrium bid functions for established models of sequential auctions.