Synthesizing Control Barrier Functions With Feasible Region Iteration for Safe Reinforcement Learning

• 成果类型: Article
• 署名作者: Yang, Yujie; Zhang, Yuhang; Zou, Wenjun; Chen, Jianyu; Yin, Yuming; Eben Li, Shengbo
• 署名单位: Tsinghua University; Tsinghua University; Zhejiang University of Technology
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3336263
• 发表日期: 2024
• 页码: 2713-2720
• 关键词: Constrained control; control barrier function; Machine Learning; Neural Networks; optimal control

摘要

Safety is a critical concern when applying reinforcement learning to real-world control problems. A widely used method for ensuring safety is to learn a control barrier function with heuristic feasibility labels that come from expert demonstrations or constraint functions. However, their forward invariant sets fall short of the maximum feasible region because of inaccurate labels. This article proposes an algorithm called feasible region iteration (FRI) that learns the maximum feasible region to generate accurate feasibility labels. The core of FRI is a constraint decay function (CDF), which comes with a self-consistency condition and naturally leads to the constraint Bellman equation. The optimal CDF, which represents the maximum feasible region, is learned through the iteration of feasible region identification and feasible region expansion. Experiment results show that our algorithm achieves near-zero constraint violations and comparable or higher performance than the baselines.