Ergodic Control of a Heterogeneous Population and Application to Electricity Pricing

• 成果类型: Article
• 署名作者: Jacquet, Quentin; van Ackooij, Wim; Alasseur, Clemence; Gaubert, Stephane
• 署名单位: Electricite de France (EDF); Inria; Centre National de la Recherche Scientifique (CNRS); Institut Polytechnique de Paris; Ecole Polytechnique
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3532178
• 发表日期: 2025
• 关键词: markov decision-processes; DISCRETE-TIME; switching costs; state; Iteration; inertia; MARKETS

摘要

We consider a control problem for a heterogeneous population composed of agents able to switch at any time between different options. The controller aims to maximize an average gain per time unit, supposing that the population is of infinite size. This leads to an ergodic control problem for a mean-field Markov decision process in which the state space is a product of simplices, and the population evolves according to controlled linear dynamics. By exploiting contraction properties of the dynamics in Hilbert's projective metric, we prove that the infinite-dimensional ergodic eigenproblem admits a solution and show that the latter is in general nonunique. This allows us to obtain optimal strategies and to quantify the gap between steady-state strategies and optimal ones. In particular, we prove in the 1-D case that there exist cyclic policies-alternating between discount and profit-taking stages-which secure a greater gain than constant-price policies. On numerical aspects, we develop a policy iteration algorithm with on-the-fly generated transitions, specifically adapted to decomposable models, leading to substantial memory savings. We finally apply our results to realistic instances coming from an electricity pricing problem encountered in the retail markets and numerically observe the emergence of cyclic promotions for sufficient inertia in customer behavior.