Relaxed Indexability and Index Policy for Partially Observable Restless Bandits

• 成果类型: Article; Early Access
• 署名作者: Liu, Keqin
• 署名单位: Xi'an Jiaotong-Liverpool University
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2022.02831
• 发表日期: 2025
• 关键词: restless multi-armed bandit; partial observation; infinite state space; relaxed indexability and index policy

摘要

This paper addresses an important class of restless multiarmed bandit (RMAB) problems that finds broad application in operations research, stochastic optimization, and reinforcement learning. There are N independent Markov processes that may be operated, observed and offer rewards. Due to the resource constraint, we can only choose a subset of M (M < N) processes to operate and accrue reward determined by the states of selected processes. We formulate the problem as a partially observable RMAB with an infinite state space and design an algorithm that achieves a near-optimal performance with low complexity. Our algorithm is based on a generalization of Whittle's original idea of indexability. Referred to as the relaxed indexability, the extended definition leads to the efficient online verifications and computations of the approximate Whittle index under the proposed algorithmic framework.