Unified smoothing approach for best hyperparameter selection problem using a bilevel optimization strategy

• 成果类型: Article
• 署名作者: Alcantara, Jan Harold; Nguyen, Chieu Thanh; Okuno, Takayuki; Takeda, Akiko; Chen, Jein-Shan
• 署名单位: RIKEN; Vietnam National University of Agriculture (VNUA); Seikei University; University of Tokyo; National Taiwan Normal University
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-024-02113-z
• 发表日期: 2025
• 页码: 479-518
• 关键词: nonsmooth

摘要

Strongly motivated from applications in various fields including machine learning, the methodology of sparse optimization has been developed intensively so far. Especially, the advancement of algorithms for solving problems with nonsmooth regularizers has been remarkable. However, those algorithms suppose that weight parameters of regularizers, called hyperparameters hereafter, are pre-fixed, but it is a crucial matter how the best hyperparameter should be selected. In this paper, we focus on the hyperparameter selection of regularizers related to the & ell;p\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _p$$\end{document} function with 0 <= 1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0