Data-Pooling Reinforcement Learning for Preventative Healthcare Intervention

• 成果类型: Article; Early Access
• 署名作者: Chen, Xinyun; Shi, Pengyi; Pu, Shanwen
• 署名单位: The Chinese University of Hong Kong, Shenzhen; Purdue University System; Purdue University; Shanghai University of Finance & Economics
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2023.03880
• 发表日期: 2025
• 关键词: preventative intervention; heterogeneous populations; Reinforcement Learning

摘要

Motivated by the emerging needs of tailored intervention for heterogeneous populations in preventative healthcare applications, we consider a multistage, dynamic decision-making problem in the online setting with unknown model parameters. To deal with the pervasive issue of small sample size in parameter estimation, we develop a novel data-pooling reinforcement learning (RL) algorithm that adaptively pools historical data, with three main innovations: (i) the weight of pooling is directly tied to the performance of the decision (measured by regret) as opposed to estimation accuracy in conventional methods; (ii) no parametric assumptions are needed between historical and current data; and (iii) requiring data-sharing only via aggregate statistics, as opposed to patient-level data. Our data-pooling algorithm framework unifies a variety of RL algorithms, and we establish a theoretical performance guarantee that quantifies the regret bound reduction from our adaptive pooling design compared with benchmarks. We substantiate the theoretical development with extensive empirical experiments via a case study in the context of postdischarge intervention to prevent unplanned readmissions, generating practical insights into healthcare management. Our algorithm enables organizations to leverage public data or published studies for patient management and supports policymakers in promoting aggregate data sharing to enhance population health outcomes.