Regression analysis: likelihood, error and entropy

• 成果类型: Article
• 署名作者: Grechuk, Bogdan; Zabarankin, Michael
• 署名单位: University of Leicester; Stevens Institute of Technology
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-018-1256-6
• 发表日期: 2019
• 页码: 145-166
• 关键词: risk

摘要

In a regression with independent and identically distributed normal residuals, the log-likelihood function yields an empirical form of the L-2-norm, whereas the normal distribution can be obtained as a solution of differential entropy maximization subject to a constraint on the L-2-norm of a random variable. The L-1-norm and the double exponential (Laplace) distribution are related in a similar way. These are examples of an inter-regenerative relationship. In fact, L-2-norm and L-1-norm are just particular cases of general error measures introduced by Rockafellar et al. (Finance Stoch 10(1):51-74, 2006) on a space of random variables. General error measures are not necessarily symmetric with respect to ups and downs of a random variable, which is a desired property in finance applications where gains and losses should be treated differently. This work identifies a set of all error measures, denoted by E, and a set of all probability density functions (PDFs) that form inter-regenerative relationships (through log-likelihood and entropy maximization). It also shows that M-estimators, which arise in robust regression but, in general, are not error measures, form inter-regenerative relationships with all PDFs. In fact, the set of M-estimators, which are error measures, coincides with E. On the other hand, M-estimators are a particular case of L-estimators that also arise in robust regression. A set of L-estimators which are error measures is identified-it contains E and the so-called trimmed L-p-norms.