Byzantine-tolerant distributed learning of finite mixture models

• 成果类型: Article; Early Access
• 署名作者: Zhang, Qiong; Tan, Yan Shuo; Chen, Jiahua
• 署名单位: Renmin University of China; National University of Singapore; University of British Columbia
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412; 1467-9868
• DOI: 10.1093/jrsssb/qkag065
• 发表日期: 2026-04-16
• 关键词: aggregation; clustering; Robust Estimation; unsupervised learning; CONVERGENCE; components; algorithm; rates

摘要

Traditional statistical methods need to be updated to work with modern distributed data storage paradigms. The split-and-conquer framework that learns models on local machines and averaging their parameter estimates is common. However, this does not work for the important problem of learning finite mixture models, because subpopulation indices on each local machine may be arbitrarily permuted (the 'label switching problem'). Earlier work proposed mixture reduction (MR) to address this issue, offering an effective and efficient solution for aligning and aggregating local mixture components. Building upon this foundation, this paper considers the additional challenge of Byzantine failure, where a fraction of local machines may transmit arbitrarily erroneous information. We introduce distance-filtered mixture reduction (DFMR), a Byzantine-tolerant framework that enhances MR by adding a distance-based filtering mechanism to identify and exclude corrupted local estimates before the MR aggregation. This integration allows DFMR to maintain MR's efficiency while achieving strong robustness against Byzantine failure. We provide theoretical justification for DFMR, proving its optimal convergence rate and asymptotic equivalence to the global maximum likelihood estimate under standard assumptions. Numerical experiments on simulated and real-world data validate the effectiveness of DFMR in achieving robust and accurate aggregation in the presence of Byzantine failure.