Maximum entropy on the mean and the Cramér rate function in statistical estimation and inverse problems: properties, models, and algorithms

• 成果类型: Article
• 署名作者: Vaisbourd, Yakov; Choksi, Rustum; Goodwin, Ariel; Hoheisel, Tim; Schonlieb, Carola-Bibiane
• 署名单位: McGill University; University of Cambridge
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-024-02189-7
• 发表日期: 2025-11
• 页码: 441-490
• 关键词: Maximum Entropy on the Mean; statistical estimation; Cram & eacute; r Rate Function; Kullback-Leibler divergence; Prior Distribution; regularization; LINEAR INVERSE PROBLEMS; Bregman Proximal Gradient; convex duality; large deviations; INFORMATION; gradient; optimization; convex

摘要

We explore a method of statistical estimation called Maximum Entropy on the Mean (MEM) which is based on an information-driven criterion that quantifies the compliance of a given point with a reference prior probability measure. At the core of this approach lies the MEM function which is a partial minimization of the Kullback-Leibler divergence over a linear constraint. In many cases, it is known that this function admits a simpler representation (known as the Cram & eacute;r rate function). Via the connection to exponential families of probability distributions, we study general conditions under which this representation holds. We then address how the associated MEM estimator gives rise to a wide class of MEM-based regularized linear models for solving inverse problems. Finally, we propose an algorithmic framework to solve these problems efficiently based on the Bregman proximal gradient method, alongside proximal operators for commonly used reference distributions. The article is complemented by a software package for experimentation and exploration of the MEM approach in applications.