Risk-Sensitive Safety Filters for Reinforcement Learning With Probabilistic Guarantees

• 成果类型: Article
• 署名作者: Lederer, Armin; Noorani, Erfaun; Baras, John S.; Hirche, Sandra
• 署名单位: Swiss Federal Institutes of Technology Domain; ETH Zurich; University System of Maryland; University of Maryland College Park; University System of Maryland; University of Maryland College Park; Technical University of Munich
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3575565
• 发表日期: 2025
• 关键词: robust-control; systems

摘要

Humans have the ability to deviate from their natural behavior when necessary, which is a cognitive process called response inhibition. Similar approaches have independently received increasing attention in recent years for ensuring the safety of control. Realized using control barrier functions or predictive safety filters, these approaches can effectively ensure the satisfaction of state constraints through an online adaptation of nominal control laws, e.g., obtained through reinforcement learning. While the focus of these realizations of inhibitory control has been on risk-neutral formulations, human studies have shown a tight link between response inhibition and risk attitude. Inspired by this insight, we propose a flexible, risk-sensitive method for inhibitory control. Our method is based on a risk-aware condition for value functions, which guarantees the satisfaction of state constraints. We propose a method for learning these value functions using common techniques from reinforcement learning and derive sufficient conditions for its success. By enforcing the derived safety conditions online using the learned value function, risk-sensitive inhibitory control is effectively achieved. The effectiveness of the developed control scheme is demonstrated in simulations.