To Engage or Not to Engage with Al for Critical Judgments: How Professionals Deal with Opacity When Using AI for Medical Diagnosis

• 成果类型: Article
• 署名作者: Lebovitz, Sarah; Lifshitz-Assaf, Hila; Levina, Natalia
• 署名单位: University of Virginia; New York University
• 刊物名称: ORGANIZATION SCIENCE
• ISSN/ISSBN: 1047-7039
• DOI: 10.1287/orsc.2021.1549
• 发表日期: 2022
• 页码: 126-148
• 关键词: Artificial intelligence; opacity; explainability; TRANSPARENCY; augmentation; technology adoption and use; uncertainty; INNOVATION; professional judgment; expertise; decision making; medical diagnosis

摘要

Artificial intelligence (AI) technologies promise to transform how professionals conduct knowledge work by augmenting their capabilities for making professional judgments. We know little, however, about how human-AI augmentation takes place in practice. Yet, gaining this understanding is particularly important when professionals use AI tools to form judgments on critical decisions. We conducted an in-depth field study in a major US. hospital where AI tools were used in three departments by diagnostic radiologists making breast cancer, lung cancer, and bone age determinations. The study illustrates the hindering effects of opacity that professionals experienced when using AI tools and explores how these professionals grappled with it in practice. In all three departments, this opacity resulted in professionals experiencing increased uncertainty because AI tool results often diverged from their initial judgment without providing underlying reasoning. Only in one department (of the three) did professionals consistently incorporate AI results into their final judgments, achieving what we call engaged augmentation. These professionals invested in AI interrogation practices-practices enacted by human experts to relate their own knowledge claims to AI knowledge claims. Professionals in the other two departments did not enact such practices and did not incorporate AI inputs into their final decisions, which we call unengaged augmentation. Our study unpacks the challenges involved in augmenting professional judgment with powerful, yet opaque, technologies and contributes to literature on AI adoption in knowledge work.