A LASSO FOR HIERARCHICAL INTERACTIONS

• 成果类型: Article
• 署名作者: Bien, Jacob; Taylor, Jonathan; Tibshirani, Robert
• 署名单位: Cornell University; Cornell University; Stanford University; Stanford University
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/13-AOS1096
• 发表日期: 2013
• 页码: 1111-1141
• 关键词: structured variable selection; regression; regularization; shrinkage; sparsity; freedom

摘要

We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting saved by the hierarchy constraint. We distinguish between parameter sparsity-the number of nonzero coefficients-and practical sparsity-the number of raw variables one must measure to make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.