Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation

• 成果类型: Article
• 署名作者: Zhou, Qing
• 署名单位: University of California System; University of California Los Angeles
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459
• DOI: 10.1080/01621459.2014.946035
• 发表日期: 2014
• 页码: 1495-1516
• 关键词: nonconcave penalized likelihood; variable selection; adaptive lasso; inference; RECOVERY; sparsity

摘要

Regularized linear regression under the l(1) penalty, such as the Lasso, has been shown to be effective in variable selection and sparse modeling. The sampling distribution of an l(1)-penalized estimator (beta) over cap is hard to determine as the estimator is defined by an optimization problem that in general can only be solved numerically and many of its components may be exactly zero. Let S be the subgradient of the (1) norm of the coefficient vector beta evaluated at (beta) over cap. We find that the joint sampling distribution of (beta) over cap and S, together called an augmented estimator, is much more tractable and has a closed-form density under a normal error distribution in both low-dimensional (p <= n) and high-dimensional (p > n) settings. Given beta and the error variance sigma(2), one may employ standard Monte Carlo methods, such as Markov chain Monte Carlo (MCMC) and importance sampling (IS), to draw samples from the distribution of the augmented estimator and calculate expectations with respect to the sampling distribution of (beta) over cap. We develop a few concrete Monte Carlo algorithms and demonstrate with numerical examples that our approach may offer huge advantages and great flexibility in studying sampling distributions in l(1)-penalized linear regression. We also establish nonasymptotic bounds on the difference between the true sampling distribution of (beta) over cap and its estimator obtained by plugging in estimated parameters, which justifies the validity of Monte Carlo simulation from an estimated sampling distribution even when p >> n -> infinity.