Asymptotic Theory for Regularized System Identification Part I: Empirical Bayes Hyperparameter Estimator

• 成果类型: Article
• 署名作者: Ju, Yue; Mu, Biqiang; Ljung, Lennart; Chen, Tianshi
• 署名单位: The Chinese University of Hong Kong, Shenzhen; Shenzhen Research Institute of Big Data; The Chinese University of Hong Kong, Shenzhen; Chinese Academy of Sciences; Academy of Mathematics & System Sciences, CAS; Linkoping University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3259977
• 发表日期: 2023
• 页码: 7224-7239
• 关键词: Asymptotic distribution; ASYMPTOTIC THEORY; empirical Bayes (EB); hyperparameter estimator; regularized least squares (RLS); Ridge Regression

摘要

Regularized techniques, also named as kernel-based techniques, are the major advances in system identification in the last decade. Although many promising results have been achieved, their theoretical analysis is far from complete and there are still many key problems to be solved. One of them is the asymptotic theory, which is about convergence properties of the model estimators as the sample size goes to infinity. The existing related results for regularized system identification are about the almost sure convergence of various hyperparameter estimators. A common problem of those results is that they do not contain information on the factors that affect the convergence properties of those hyperparameter estimators, e.g., the regression matrix. In this article, we tackle problems of this kind for the regularized finite impulse response model estimation with the empirical Bayes (EB) hyperparameter estimator and filtered white noise input. In order to expose and find those factors, we study the convergence in distribution of the EB hyperparameter estimator, and the asymptotic distribution of its corresponding model estimator. For illustration, we run Monte Carlo simulations to show the efficacy of our obtained theoretical results.