Generalized Simultaneous Perturbation-Based Gradient Search With Reduced Estimator Bias

• 成果类型: Article
• 署名作者: Pachal, Soumen; Bhatnagar, Shalabh; Prashanth, L. A.
• 署名单位: Indian Institute of Technology System (IIT System); Indian Institute of Technology (IIT) - Madras; Indian Institute of Science (IISC) - Bangalore; Indian Institute of Science (IISC) - Bangalore; Bosch; Indian Institute of Technology System (IIT System); Indian Institute of Technology (IIT) - Madras
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3532160
• 发表日期: 2025
• 页码: 4687-4702
• 关键词: convergence; Perturbation methods; Approximation algorithms; gradient methods; vectors; Noise measurement; accuracy; Taylor series; computer science; estimation; Balanced generalized simultaneous perturbation stochastic approximation (B-GSPSA); generalized random directions stochastic approximation (GRDSA); generalized simultaneous perturbation-based gradient search (GSPGS); generalized smoothed functional (GSF) procedure; GSPSA; stochastic optimization

摘要

We present a family of generalized simultaneous perturbation-based gradient search (GSPGS) estimators that use noisy function measurements. The number of function measurements required by each estimator is guided by the desired level of accuracy. We first present in detail unbalanced generalized simultaneous perturbation stochastic approximation estimators and later present the balanced versions of these. We extend this idea further and present the generalized smoothed functional and generalized random directions stochastic approximation estimators, respectively, as well as their balanced variants. We show that estimators within any specified class requiring more number of function measurements result in lower estimator bias. We present a detailed analysis of both the asymptotic and nonasymptotic convergence of the resulting stochastic approximation schemes. We further present a series of experimental results with the various GSPGS estimators on the Rastrigin and quadratic function objectives. Our experiments are seen to validate our theoretical findings.