Improving Human Sequential Decision Making with Reinforcement Learning

• 成果类型: Article; Early Access
• 署名作者: Bastani, Hamsa; Bastani, Osbert; Sinchaisri, Wichinpong Park
• 署名单位: University of Pennsylvania; University of Pennsylvania; University of California System; University of California Berkeley
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2022.02455
• 发表日期: 2025
• 关键词: Behavioral Operations; interpretable reinforcement learning; sequential decision making; human-AI interface

摘要

Workers spend a significant amount of time learning how to make good decisions. Evaluating the efficacy of a given decision, however, can be complicated-for example, decision outcomes are often long-term and relate to the original decision in complex ways. Surprisingly, even though learning good decision-making strategies is difficult, the strategies can often be expressed in simple and concise forms. Focusing on sequential decision making, we design a novel machine learning algorithm that is capable of extracting best practices from trace data and conveying its insights to humans in the form of interpretable tips. Our algorithm selects the tip that best bridges the gap between the actions taken by human workers and those taken by the optimal policy in a way that accounts for which actions are consequential for achieving higher performance. We evaluate our approach through a series of randomized controlled experiments where participants manage a virtual kitchen. Our experiments show that the tips generated by our algorithm can significantly improve human performance relative to intuitive baselines. In addition, we discuss a number of empirical insights that can help inform the design of algorithms intended for human-AI interfaces. For instance, we find evidence that participants do not simply blindly follow our tips; instead, they combine them with their own experience to discover additional strategies for improving performance.