Sensitivity to Distance and Baseline Distributions in Forecast Evaluation

• 成果类型: Article
• 署名作者: Jose, Victor Richmond R.; Nau, Robert F.; Winkler, Robert L.
• 署名单位: Duke University
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.1080.0955
• 发表日期: 2009
• 页码: 582-590
• 关键词: forecast verification; ranked categories; Scoring rules; sensitivity to distance; baseline distributions

摘要

Scoring rules can provide incentives for truthful reporting of probabilities and evaluation measures for the probabilities after the events of interest are observed. Often the space of events is ordered and an evaluation relative to some baseline distribution is desired. Scoring rules typically studied in the literature and used in practice do not take account of any ordering of events, and they evaluate probabilities relative to a default baseline distribution. In this paper, we construct rich families of scoring rules that are strictly proper (thereby encouraging truthful reporting), are sensitive to distance (thereby taking into account ordering of events), and incorporate a baseline distribution relative to which the value of a forecast is measured. In particular, we extend the power and pseudospherical families of scoring rules to allow for sensitivity to distance, with or without a specified baseline distribution.