Difference of convex algorithms for bilevel programs with applications in hyperparameter selection

• 成果类型: Article
• 署名作者: Ye, Jane J.; Yuan, Xiaoming; Zeng, Shangzhi; Zhang, Jin
• 署名单位: University of Victoria; University of Hong Kong; Peng Cheng Laboratory; Southern University of Science & Technology
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-022-01888-3
• 发表日期: 2023
• 页码: 1583-1616
• 关键词: model selection; dca

摘要

In this paper, we present difference of convex algorithms for solving bilevel programs in which the upper level objective functions are difference of convex functions, and the lower level programs are fully convex. This nontrivial class of bilevel programs provides a powerful modelling framework for dealing with applications arising from hyperparameter selection in machine learning. Thanks to the full convexity of the lower level program, the value function of the lower level program turns out to be convex and hence the bilevel program can be reformulated as a difference of convex bilevel program. We propose two algorithms for solving the reformulated difference of convex program and show their convergence to stationary points under very mild assumptions. Finally we conduct numerical experiments to a bilevel model of support vector machine classification.