Benign Nonconvex Landscapes in Optimal and Robust Control, Part I: Global Optimality

• 成果类型: Article
• 署名作者: Zheng, Yang; Pai, Chih-Fan Rich; Tang, Yujie
• 署名单位: University of California System; University of California San Diego; Peking University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2026.3675544
• 发表日期: 2026
• 关键词: LINEAR-QUADRATIC REGULATOR; OUTPUT-FEEDBACK CONTROL; policy optimization; nonsmooth optimization; gradient methods; h-2

摘要

Direct policy search has achieved great empir-ical success in reinforcement learning. Many recent stud-ies have revisited its theoretical foundation for continuous control, which reveals elegant nonconvex geometry in var-ious benchmark problems. This article considers two fun-damental optimal and robust control problems with partial observability: linear quadratic Gaussian (LQG) control and H infinity robust control. In the policy space, the former problem is smooth but nonconvex, while the latter one is nonsmooth and nonconvex. We highlight some interesting and surpris-ing discontinuity of LQG and H infinity cost functions around the boundary of their domains. Despite the lack of convex-ity (and possibly smoothness), we show that for a class of nondegenerate policies, all Clarke stationary points are globally optimal and there is no spurious local minimum for both LQG and H infinity control. The main results are established by a new and unified framework of Extended Convex Lifting (ECL), which reconciles the gap between nonconvex policy optimization and convex reformulations. This ECL frame-work is of independent interest, and we discuss its details in Part II of this article