A BERNSTEIN-TYPE INEQUALITY FOR SOME MIXING PROCESSES AND DYNAMICAL SYSTEMS WITH AN APPLICATION TO LEARNING

• 成果类型: Article
• 署名作者: Hang, Hanyuan; Steinwart, Ingo
• 署名单位: University of Stuttgart
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/16-AOS1465
• 发表日期: 2017
• 页码: 708-743
• 关键词: exponential inequalities; STATISTICAL PROPERTIES; devroye inequality; decay; MAPS; CONVERGENCE; THEOREM; points

摘要

We establish a Bernstein-type inequality for a class of stochastic processes that includes the classical geometrically phi-mixing processes, Rio's generalization of these processes and many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d. data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support vector machines using the Gaussian kernels for binary classification, we obtain essentially the same rate as for i.i.d. processes, and for least squares and quantile regression; it turns out that the resulting learning rates match, up to some arbitrarily small extra term in the exponent, the optimal rates for i.i.d. processes.