GUARDRAILS FOR HUMAN-AI ECOLOGIES: NORM-BASED COORDINATION AND DESIGN FOR PREDICTABILITY

• 成果类型: Article
• 署名作者: Grisold, Thomas; Berente, Nicholas; Seidel, Stefan
• 署名单位: Vienna University of Economics & Business; University of St Gallen; University of Notre Dame; University of Cologne
• 刊物名称: MIS QUARTERLY
• ISSN/ISSBN: 0276-7783
• DOI: 10.25300/MISQ/2025/18058
• 发表日期: 2025-12
• 页码: 1239-1266
• 关键词: Human-AI ecology; unstructured coordination; social norms; predictive processing; design principles; RELATIONAL COORDINATION; INPUT UNCERTAINTY; TECHNOLOGY; mechanisms; systems; FUTURE

摘要

Human-AI ecologies involve human and AI-based agents that coordinate their interactions in part by following social norms. Social norms, therefore, are important for establishing the guardrails that ensure desirable interactions in a way that is consistent with essential values, such as human safety. Managing human-AI ecologies requires specifying norms to enable coordination in known situations but also allowing for the emergence of norms to enable coordination in unspecified, unstructured situations. We integrate predictive processing theory and social norm theory to explain how existing norms are enacted and reinforced based on agents' predictive models and how new norms emerge as agents update their predictive models in response to prediction errors in uncertain coordination scenarios. Rooted in this perspective, we develop a design theory that emphasizes design for predictability and propose a set of design principles for managers and developers to encode norms to evolve in human-AI ecologies, monitor outcomes, and intervene when necessary.