Effective degrees of freedom: a flawed metaphor

• 成果类型: Article
• 署名作者: Janson, Lucas; Fithian, William; Hastie, Trevor J.
• 署名单位: Stanford University
• 刊物名称: BIOMETRIKA
• ISSN/ISSBN: 0006-3444
• DOI: 10.1093/biomet/asv019
• 发表日期: 2015
• 页码: 479485
• 关键词: prediction rule; ERROR RATE

摘要

To most applied statisticians, a fitting procedure's degrees of freedom is synonymous with its model complexity, or its capacity for overfitting to data. In particular, the degrees of freedom is often used to parameterize the bias-variance trade-off in model selection. We argue that, on the contrary, model complexity and degrees of freedom may correspond very poorly. We exhibit and theoretically explore various fitting procedures for which the degrees of freedom is not monotonic in the model complexity parameter and can exceed the total dimension of the ambient space even in very simple settings. We show that the degrees of freedom for any nonconvex projection method can be unbounded.