Engression: extrapolation through the lens of distributional regression

• 成果类型: Article
• 署名作者: Shen, Xinwei; Meinshausen, Nicolai
• 署名单位: Swiss Federal Institutes of Technology Domain; ETH Zurich
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1093/jrsssb/qkae108
• 发表日期: 2025
• 页码: 653-677
• 关键词: Quantile regression; Scoring rules; prediction; statistics; CURVES

摘要

Distributional regression aims to estimate the full conditional distribution of a target variable, given covariates. Popular methods include linear and tree ensemble based quantile regression. We propose a neural network- based distributional regression methodology called 'engression'. An engression model is generative in the sense that we can sample from the fitted conditional distribution and is also suitable for high-dimensional outcomes. Furthermore, we find that modelling the conditional distribution on training data can constrain the fitted function outside of the training support, which offers a new perspective to the challenging extrapolation problem in nonlinear regression. In particular, for 'preadditive noise' models, where noise is added to the covariates before applying a nonlinear transformation, we show that engression can successfully perform extrapolation under some assumptions such as monotonicity, whereas traditional regression approaches such as least-squares or quantile regression fall short under the same assumptions. Our empirical results, from both simulated and real data, validate the effectiveness of the engression method. The software implementations of engression are available in both R and Python.