Trimmed Opinion Pools and the Crowd's Calibration Problem

• 成果类型: Article
• 署名作者: Jose, Victor Richmond R.; Grushka-Cockayne, Yael; Lichtendahl, Kenneth C., Jr.
• 署名单位: Georgetown University; University of Virginia
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2013.1781
• 发表日期: 2014
• 页码: 463-475
• 关键词: trimming; probability forecasts; expert combination; linear opinion pool; UNDERCONFIDENCE; overconfidence; Scoring rules; Wisdom of crowds; diversity

摘要

We introduce an alternative to the popular linear opinion pool for combining individual probability forecasts. One of the well-known problems with the linear opinion pool is that it can be poorly calibrated. It tends toward underconfidence as the crowd's diversity increases, i.e., as the variance in the individuals' means increases. To address this calibration problem, we propose the exterior-trimmed opinion pool. To form this pool, forecasts with low and high means, or cumulative distribution function (cdf) values, are trimmed away from a linear opinion pool. Exterior trimming decreases the pool's variance and improves its calibration. A linear opinion pool, however, will remain overconfident when individuals are overconfident and not very diverse. For these situations, we suggest trimming away forecasts with moderate means or cdf values. This interior trimming increases variance and reduces overconfidence. Using probability forecast data from U.S. and European Surveys of Professional Forecasters, we present empirical evidence that trimmed opinion pools can outperform the linear opinion pool.