An analysis of the EM algorithm and entropy-like proximal point methods

• 成果类型: Article
• 署名作者: Tseng, P
• 署名单位: University of Washington; University of Washington Seattle
• 刊物名称: MATHEMATICS OF OPERATIONS RESEARCH
• ISSN/ISSBN: 0364-765X
• DOI: 10.1287/moor.1030.0073
• 发表日期: 2004
• 页码: 27-44
• 关键词: multiplier methods; CONVERGENCE; convex

摘要

The EM algorithm is a popular method for maximum likelihood estimation from incomplete. data. This method may be viewed as a proximal point method for maximizing the log-likelihood function using an integral form of the Kullback-Leibler distance function. Motivated by this interpretation, we consider a proximal point method using an integral form of entropy-like distance function. We give a convergence analysis of the resulting proximal point method in the case where the cluster points lie in the interior of the objective function domain. This result is applied to a normal/independent example and a Gaussian mixture example to establish convergence of the EM algorithm on these examples. Further convergence analysis of the method for maximization over an orthant is given in low dimensions. Sublinear convergence and schemes for accelerating convergence are also discussed.