Differentially Private Distributed Nonconvex Stochastic Optimization With Quantized Communication

• 成果类型: Article
• 署名作者: Chen, Jialong; Wang, Jimin; Zhang, Ji-Feng
• 署名单位: Chinese Academy of Sciences; Nanjing Institute of Geology & Paleontology, CAS; Academy of Mathematics & System Sciences, CAS; Chinese Academy of Sciences; University of Chinese Academy of Sciences, CAS; University of Science & Technology Beijing; Zhongyuan University of Technology
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3590872
• 发表日期: 2026

摘要

This article proposes a novel distributed nonconvex stochastic optimization algorithm that can achieve privacy protection and convergence simultaneously while improving communication efficiency. Specifically, each node adds general privacy noises to its local state to avoid information leakage, and then, quantizes its noise-perturbed state before transmitting to improve communication efficiency. By using a sampling parameter-controlled subsampling method, the proposed algorithm enhances the differential privacy level compared to the existing works. By using a new convergence analysis technique, the mean square convergence for nonconvex cost functions is given without assuming that gradients are bounded. Furthermore, when the nonconvex cost function satisfies the Polyak- & Lstrok;ojasiewicz condition, a convergence rate and the oracle complexity of the proposed algorithm are given. By using a two-time-scale step-sizes method and a probabilistic quantizer, the proposed algorithm achieves finite cumulative differential privacy budgets is an element of , delta and the mean square convergence simultaneously while improving communication efficiency as the sample-size goes to infinity. A numerical example of the distributed training on the MNIST dataset is given to show the effectiveness and advantages of the algorithm.