Is Separately Modeling Subpopulations Beneficial for Sequential Decision-Making?

• 成果类型: Article
• 署名作者: Lee, Ilbin
• 署名单位: University of Alberta
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2023.2474
• 发表日期: 2024
• 页码: 2595-2611
• 关键词: models; dynamic programming; Treatment; Healthcare; estimation; statistics

摘要

In recent applications of Markov decision processes (MDPs), it is common to estimate transition probabilities and rewards from transition data. In healthcare and some other applications, transition data are collected from a population of different entities, such as patients. Thus, one faces a modeling question of whether to estimate different models for subpopulations (e.g., divided by smoking status). For instance, there may be a subpopulation whose disease status progresses faster than others, and for such a group, estimating a separate model and applying the corresponding optimal treatment plan can improve their outcomes. This work provides theoretical results and empirical methods for making the decision of whether to model subpopulations separately (called stratifying) or not. We also present how to use our results to select the best stratification among many. We illustrate our results and methods numerically using random instances and a medical decision-making problem from the literature. Because improving medical decisions by tailoring to each subpopulation is a building block of precision medicine, this work advances the use of MDPs in medical decision-making toward the precision medicine paradigm.