Least Squares Monte Carlo and Pathwise Optimization for Merchant Energy Production

• 成果类型: Article
• 署名作者: Yang, Bo; Nadarajah, Selvaprabu; Secomandi, Nicola
• 署名单位: Carnegie Mellon University; University of Illinois System; University of Illinois Chicago; University of Illinois Chicago Hospital
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2018.0341
• 发表日期: 2024
• 页码: 2758-2775
• 关键词: block coordinate descent; information relaxation and duality; least squares Monte Carlo; Markov decision processes; merchant energy operations; pathwise optimization; Principal Component Analysis; Real options; Reinforcement Learning

摘要

We study merchant energy production modeled as a compound switching and timing option. The resulting Markov decision process is intractable. Least squares Monte Carlo combined with information relaxation and duality is a state-of-the-art reinforcement learning methodology to obtain operating policies and optimality gaps for related models. Pathwise optimization is a competing technique developed for optimal stopping settings, in which it typically provides superior results compared with this approach, albeit with a larger computational effort. We apply these procedures to merchant energy production. Using pathwise optimization requires methodological extensions. We use principal component analysis and block coordinate descent in novel ways to respectively precondition and solve the ensuing ill-conditioned and large-scale linear program, which even a cutting-edge commercial solver is unable to handle directly. Both techniques yield near optimal operating policies on realistic ethanol production instances. However, at the cost of both considerably longer run times and greater memory usage, which limits the number of stages of the instances that it can handle, pathwise optimization leads to substantially tighter dual bounds compared with least squares Monte Carlo, even when specified in a simple fashion, complementing it in this case. Thus, it plays a critical role in obtaining small optimality gaps. Our numerical observations on the magnitudes of these bound improvements differ from what is currently known. This research has potential relevance for other commodity merchant operations contexts and motivates additional algorithmic work in the area of pathwise optimization.