On Asymptotic Optimality of Cross-Validation Estimators for Kernel-Based Regularized System Identification

• 成果类型: Article
• 署名作者: Mu, Biqiang; Chen, Tianshi
• 署名单位: Chinese Academy of Sciences; The Chinese University of Hong Kong, Shenzhen; Shenzhen Research Institute of Big Data; The Chinese University of Hong Kong, Shenzhen
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3322576
• 发表日期: 2024
• 页码: 4352-4367
• 关键词: Kernel; estimation; Parameter Estimation; Finite impulse response filters; CONTRACTS; training; systematics; Asymptotic optimality (AO); cross-validation (CV) methods; finite impulse response; regularized least squares estimators

摘要

Kernel-based regularized system identification is one of the major advances in system identification in the past decade. A recent focus is to develop its asymptotic theory and it has been found that the Stein's unbiased risk estimator is asymptotically optimal (AO) in the sense of minimizing the mean squared error for prediction ability, but the empirical Bayes estimator is not AO in general. In this article, we further study the AO of various cross-validation (CV) estimators and show that the generalized CV method, leave k-out CV method, and r-fold CV method are all AO under mild assumptions, but the hold out CV method is not AO in general. We illustrate the efficacy of our theoretical results through numerical simulations.