Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs

• 成果类型: Article
• 署名作者: Grushka-Cockayne, Yael; Lichtendahl, Kenneth C., Jr.; Jose, Victor Richmond R.; Winkler, Robert L.
• 署名单位: University of Virginia; Georgetown University; Duke University
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2017.1588
• 发表日期: 2017
• 页码: 712-728
• 关键词: SCORING RULES; Probability elicitation

摘要

From forecasting competitions to conditional value-at-risk requirements, the use of multiple quantile assessments is growing in practice. To evaluate them, we use a rule from the general class of proper scoring rules for a forecaster's multiple quantiles of a single uncertain quantity of interest. The general rule is additive in the component scores. Each component contains a function that measures its quantile's distance from the realization and weights its contribution to the overall score. To determine this function, we propose that the score of a group's combined quantile should be better than that of a randomly selected forecaster's quantile only when the forecasters bracket the realization (i.e., their quantiles do not fall on the same side of the realization). If a score satisfies this property, we say it is sensitive to bracketing. We characterize the class of proper scoring rules that is sensitive to bracketing when the decision maker uses a generalized average to combine forecasters' quantiles. Finally, we show how weights can be set to match the payoffs in many important business contexts.