WHY LEAST-SQUARES AND MAXIMUM-ENTROPY - AN AXIOMATIC APPROACH TO INFERENCE FOR LINEAR INVERSE PROBLEMS

• 成果类型: Article
• 署名作者: CSISZAR, I
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/aos/1176348385
• 发表日期: 1991
• 页码: 2032-2066
• 关键词: incomplete-data; likelihood

摘要

An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all n-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a prior guess is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's f-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained.