Deep Mutual Density Ratio Estimation with Bregman Divergence and Its Applications

• 成果类型: Article
• 署名作者: Han, Dongxiao; Zheng, Siming; Shen, Guohao; Song, Xinyuan; Sun, Liuquan; Huang, Jian
• 署名单位: Nankai University; Nankai University; Chinese University of Hong Kong; Hong Kong Polytechnic University; Chinese Academy of Sciences; Nanjing Institute of Geology & Paleontology, CAS; Academy of Mathematics & System Sciences, CAS; Hong Kong Polytechnic University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2025.2507437
• 发表日期: 2025-07-03
• 页码: 1990-2001
• 关键词: Bregman divergence; Conditional probability density; Deep neural network; Mutual density ratio; Mutual information; neural-networks; dependence; regression

摘要

This article introduces a unified approach to estimating the mutual density ratio, defined as the ratio between the joint density function and the product of the individual marginal density functions of two random vectors. It serves as a fundamental measure for quantifying the relationship between two random vectors. Our method uses the Bregman divergence to construct the objective function and leverages deep neural networks to approximate the logarithm of the mutual density ratio. We establish a non-asymptotic error bound for our estimator, achieving the optimal minimax rate of convergence under a bounded support condition. Additionally, our estimator mitigates the curse of dimensionality when the distribution is supported on a lower-dimensional manifold. We extend our results to overparameterized neural networks and the case with unbounded support. Applications of our method include conditional probability density estimation, mutual information estimation, and independence testing. Simulation studies and real data examples demonstrate the effectiveness of our approach. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.