Annealing Optimization for Progressive Learning With Stochastic Approximation

• 成果类型: Article
• 署名作者: Mavridis, Christos N.; Baras, John S.
• 署名单位: University System of Maryland; University of Maryland College Park; University System of Maryland; University of Maryland College Park
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2022.3232706
• 发表日期: 2023
• 页码: 2862-2874
• 关键词: approximation algorithms; optimization; Heuristic algorithms; Annealing; Machine learning algorithms; dynamical systems; Complexity theory; Optimization for machine learning; annealing optimization; online deterministic annealing (ODA); progressive learning; reinforcement learning (RL); stochastic approximation

摘要

In this work, we introduce a learning model designed to meet the needs of applications in which computational resources are limited, and robustness and interpretability are prioritized. Learning problems can be formulated as constrained stochastic optimization problems, with the constraints originating mainly from model assumptions that define a tradeoff between complexity and performance. This tradeoff is closely related to overfitting, generalization capacity, and robustness to noise and adversarial attacks, and depends on both the structure and complexity of the model, as well as the properties of the optimization methods used. We develop an online prototype-based learning algorithm based on annealing optimization that is formulated as an online gradient-free stochastic approximation algorithm. The learning model can be viewed as an interpretable and progressively growing competitive-learning neural network model to be used for supervised, unsupervised, and reinforcement learning. The annealing nature of the algorithm contributes to minimal hyperparameter tuning requirements, poor local minima prevention, and robustness with respect to the initial conditions. At the same time, it provides online control over the performance-complexity tradeoff by progressively increasing the complexity of the learning model as needed, through an intuitive bifurcation phenomenon. Finally, the use of stochastic approximation enables the study of the convergence of the learning algorithm through mathematical tools from dynamical systems and control, and allows for its integration with reinforcement learning algorithms, constructing an adaptive state-action aggregation scheme.