When Emotion AI Meets Strategic Users

• 成果类型: Article; Early Access
• 署名作者: Yu, Yifan; Xue, Wendao; Jia, Lin; Tan, Yong
• 署名单位: University of Texas System; University of Texas Austin; Beijing Institute of Technology; Beijing Institute of Technology; University of Washington; University of Washington Seattle
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2022.02860
• 发表日期: 2025
• 关键词: emotion AI; strategic users; signaling game; muddled information

摘要

When organizations adopt artificial intelligence (AI) to recognize individuals' negative emotions and accordingly allocate limited resources, strategic users are incentivized to game the system by misrepresenting their emotions. The value of AI in automating such emotion-driven allocation may be undermined by gaming behavior, algorithmic noise in emotion detection, and the spillover effect of negative emotions. We develop a gametheoretical model to understand emotion AI adoption, particularly in customer care, and analyze the design of the associated allocation policies. We find that adopting emotion AI is valuable if the spillover effect of negative emotions is negligible compared with resource misallocation loss, regardless of algorithmic noise and gaming behavior. We also quantify the welfare impacts of emotion AI on the users, organization, and society. Notably, a stronger AI is not always socially desirable and regulation on emotion-driven allocation is needed. Finally, we characterize conditions under which leveraging the AI system is preferred to hiring human employees in emotion-driven allocation. We also explore the alternative application of using emotion AI to monitor strategic employees and compare it with hiring a human manager for monitoring. Intriguingly, algorithmic noise may increase the profit of AI monitoring. Our work provides implications for designing, adopting, and regulating emotion AI.