ONLINE INFERENCE WITH MULTI-MODAL LIKELIHOOD FUNCTIONS
成果类型：
Article
署名作者：
Gerber, Mathieu; Heine, Kari
署名单位：
University of Bristol; University of Bath
刊物名称：
ANNALS OF STATISTICS
ISSN/ISSBN：
0090-5364
DOI：
10.1214/21-AOS2076
发表日期：
2021
页码：
3103-3126
关键词：
摘要：
Let (Y-t)(t >= 1) be a sequence of i.i.d. observations and {f(theta), theta is an element of R-d} be a parametric model. We introduce a new online algorithm for computing a sequence ((theta) over cap (t))(t >= 1), which is shown to converge almost surely to argmax(theta is an element of Rd) E[log f(theta)(Y-1)] at rate O(log(t)((1+epsilon)/2t-1/2)), with epsilon > 0 a user specified parameter. This convergence result is obtained under standard conditions on the statistical model and, most notably, we allow the mapping theta bar right arrow E[log f(theta)(Y-1)] to be multi-modal. However, the computational cost to process each observation grows exponentially with the dimension of theta, which makes the proposed approach applicable to low or moderate dimensional problems only. We also derive a version of the estimator (theta) over cap (t), which is well suited to student-t linear regression models that are popular tools for robust linear regression. As shown by experiments on simulated and real data, the corresponding estimator of the regression coefficients is, as expected, robust to the presence of outliers and thus, as a by-product, we obtain a new adaptive and robust online estimation procedure for linear regression models.