Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise

• 成果类型: Article
• 署名作者: Yaji, Vinayaka G.; Bhatnagar, Shalabh
• 署名单位: Indian Institute of Science (IISC) - Bangalore
• 刊物名称: MATHEMATICS OF OPERATIONS RESEARCH
• ISSN/ISSBN: 0364-765X
• DOI: 10.1287/moor.2019.1037
• 发表日期: 2020
• 页码: 1405-1444
• 关键词: Approximation; CONVERGENCE

摘要

In this paper, we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of non-additive iterate-dependent Markov noise. We show that the recursion on each timescale tracks the flow of a differential inclusion obtained by averaging the set-valued drift function in the recursion with respect to a set of measures accounting for both averaging with respect to the stationary distributions of the Markov noise terms and the interdependence between the two recursions on different timescales. The framework studied in this paper builds on a recent work by Ramaswamy and Bhatnagar, by allowing for the presence of nonadditive iterate-dependent Markov noise. As an application, we consider the problem of computing the optimum in a constrained convex optimization problem, where the objective function and the constraints are averaged with respect to the stationary distribution of an underlying Markov chain. Further, the proposed scheme neither requires the differentiability of the objective function nor the knowledge of the averaging measure.