Learning-Based Probabilistic LTL Motion Planning With Environment and Motion Uncertainties

• 成果类型: Article
• 署名作者: Cai, Mingyu; Peng, Hao; Li, Zhijun; Kan, Zhen
• 署名单位: University of Iowa; Chinese Academy of Sciences; University of Science & Technology of China, CAS
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2020.3006967
• 发表日期: 2021
• 页码: 2386-2392
• 关键词: uncertainty; Probabilistic logic; Task analysis; PLANNING; Learning automata; Markov processes; Autonomous agents; linear temporal logic (LTL); Markov decision process (MDP); motion planning; Reinforcement Learning

摘要

This article considers control synthesis of an autonomous agent with linear temporal logic (LTL) specifications subject to environment and motion uncertainties. Specifically, the probabilistic motion of the agent is modeled by a Markov decision process (MDP) with unknown transition probabilities. The operating environment is assumed to be partially known, where the desired LTL specifications might be partially infeasible. A relaxed product MDP is constructed that allows the agent to revise its motion plan without strictly following the desired LTL constraints. A utility function composed of violation cost and state rewards is developed. Rigorous analysis shows that, if there almost surely (i.e., with probability 1) exists a policy that satisfies the relaxed product MDP, any algorithm that optimizes the expected utility is guaranteed to find such a policy. A reinforcement learning-based approach is then developed to generate policies that fulfill the desired LTL specifications as much as possible by optimizing the expected discount utility of the relaxed product MDP.