Contextual Learning with Online Convex Optimization: Theory and Application to Medical Decision-Making

• 成果类型: Article; Early Access
• 署名作者: Keyvanshokooh, Esmaeil; Zhalechian, Mohammad; Shi, Cong; Van Oyen, Mark P.; Kazemian, Pooyan
• 署名单位: Texas A&M University System; Texas A&M University College Station; Mays Business School; Indiana University System; IU Kelley School of Business; Indiana University Bloomington; University of Miami; University of Michigan System; University of Michigan; University System of Ohio; Case Western Reserve University
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2019.03211
• 发表日期: 2025
• 关键词: contextual bandits; online convex optimization; online learning; personalized medicine; regret

摘要

Optimizing the treatment regimen is a fundamental medical decision-making problem. This can be thought of as a two-dimensional decision-making problem with a nested structure because it involves determining both the optimal medication and its optimal dose. Identifying the most effective medication for an individual often poses considerable difficulty, and even when a suitable medication is ascertained, dosing it optimally remains a significant challenge. Making these two nested decisions necessitates the adaptive learning of a personalized disease progression control model. To address this problem, we propose a novel contextual multiarmed bandit model under a two-dimensional control with a nested structure. For this model, we develop a new joint contextual learning and optimization algorithm, termed the stochastic subgradient descent atop contextual multiarmed bandit (SGD-MAB) algorithm. It sequentially selects for a patient (i) the best medication based on their contextual information and (ii) the corresponding dose optimized over the prior history of those patients who received the same medication. We prove that it admits a sublinear regret, which is tight up to a logarithmic factor. Our regret analysis leverages the strengths of both contextual bandit approaches and online convex optimization techniques in a seamless fashion. We substantiate the practicality of SGD-MAB using clinical data on patients with hypertension and heightened cardiovascular risks. Our analysis indicates that SGD-MAB has the potential to surpass current practices. We benchmark several policies to show the advantages of our approach and offer critical insights. Our framework holds promise for various applications beyond healthcare that require nested decision-making.