EMPIRICAL RISK MINIMIZATION AND COMPLEXITY OF DYNAMICAL MODELS

• 成果类型: Article
• 署名作者: McGoff, Kevin; Nobel, Andrew B.
• 署名单位: University of North Carolina; University of North Carolina Charlotte; University of North Carolina; University of North Carolina Chapel Hill
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/19-AOS1876
• 发表日期: 2020
• 页码: 2031-2054
• 关键词: prediction; entropy; CONVERGENCE; systems; bounds

摘要

A dynamical model consists of a continuous self-map T : chi -> chi of a compact state space chi and a continuous observation function f : chi -> R. This paper considers the fitting of a parametrized family of dynamical models to an observed real-valued stochastic process using empirical risk minimization. The limiting behavior of the minimum risk parameters is studied in a general setting. We establish a general convergence theorem for minimum risk estimators and ergodic observations. We then study conditions under which empirical risk minimization can effectively separate signal from noise in an additive observational noise model. The key condition in the latter results is that the family of dynamical models has limited complexity, which is quantified through a notion of entropy for families of infinite sequences that connects covering number based entropies with topological entropy studied in dynamical systems. We establish close connections between entropy and limiting average mean widths for stationary processes, and discuss several examples of dynamical models.