The ODE method for stability of skip-free Markov chains with applications to MCMC

• 成果类型: Article
• 署名作者: Fort, Gersende; Meyn, Sean; Moulines, Eric; Priouret, Pierre
• 署名单位: IMT - Institut Mines-Telecom; Institut Polytechnique de Paris; Telecom Paris; Centre National de la Recherche Scientifique (CNRS); University of Illinois System; University of Illinois Urbana-Champaign; University of Illinois System; University of Illinois Urbana-Champaign; Sorbonne Universite; Universite Paris Cite
• 刊物名称: ANNALS OF APPLIED PROBABILITY
• ISSN/ISSBN: 1050-5164
• DOI: 10.1214/07-AAP471
• 发表日期: 2008
• 页码: 664-707
• 关键词: multiclass queuing-networks; Performance evaluation; metropolis algorithms; subgeometric rates; CONVERGENCE; models; instability; ergodicity; workload

摘要

In this paper, some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space and the initial condition by a large constant. The resulting fluid limit is the solution of an ordinary differential equation (ODE) in most of the state space. Stability and finer ergodic properties for the stochastic model then follow from stability of the set of fluid limits. Moreover, similarly to the queueing context where fluid models are routinely used to design control policies, the structure of the limiting ODE in this general setting provides an understanding of the dynamics of the Markov chain. These results are illustrated through application to Markov chain Monte Carlo methods.