Calibeating: Beating forecasters at their own game

• 成果类型: Article
• 署名作者: Foster, Dean P.; Hart, Sergiu
• 署名单位: University of Pennsylvania; Amazon.com; Hebrew University of Jerusalem; Hebrew University of Jerusalem
• 刊物名称: THEORETICAL ECONOMICS
• ISSN/ISSBN: 1933-6837
• DOI: 10.3982/TE5330
• 发表日期: 2023-11-01
• 页码: 1441-1474
• 关键词: Calibrated forecasts; calibeating; Brier score; calibration score; refinement score; experts; C1; c7; D8

摘要

To identify expertise, forecasters should not be tested by their calibration score, which can always be made arbitrarily small, but rather by their Brier score. The Brier score is the sum of the calibration score and the refinement score; the latter measures how good the sorting into bins with the same forecast is, and thus attests to expertise. This raises the question of whether one can gain calibration without losing expertise, which we refer to as calibeating. We provide an easy way to calibeat any forecast, by a deterministic online procedure. We moreover show that calibeating can be achieved by a stochastic procedure that is itself calibrated, and then extend the results to simultaneously calibeating multiple procedures, and to deterministic procedures that are continuously calibrated.