A sequence of predictions is calibrated if and only if it induces no swap regret to all down-stream decision tasks. We study the Maximum Swap Regret (MSR) of predictions for binary events: the swap regret maximized over all downstream tasks with bounded payoffs. Previously, the best online prediction algorithm for minimizing MSR is obtained by minimizing the K1 calibration error, which upper bounds MSR up to a constant factor. However, recent work (Qiao and Valiant, 2021) gives an ${\Omega}(T^{0.528})$ lower bound for the worst-case expected K1 calibration error incurred by any randomized algorithm in T rounds, presenting a barrier to achieving better rates for MSR. Several relaxations of MSR have been considered to overcome this barrier, via external regret (Kleinberg et al., 2023) and regret bounds depending polynomially on the number of actions in downstream tasks (Noarov et al., 2023; Roth and Shi, 2024). We show that the barrier can be surpassed without any relaxations: we give an efficient randomized prediction algorithm that guarantees $O(TlogT)$ expected MSR. We also discuss the economic utility of calibration by viewing MSR as a decision-theoretic calibration error metric and study its relationship to existing metrics.
翻译:摘要:当且仅当一组预测序列对所有下游决策任务不诱导交换遗憾时,该序列才是校准的。我们研究二元事件预测的最大交换遗憾(MSR):在所有具有有界收益的下游任务上最大化的交换遗憾。此前,最小化MSR的最优在线预测算法是通过最小化K1校准误差实现的,该误差可将MSR上界控制在常数因子范围内。然而,近期研究(Qiao and Valiant, 2021)表明,任何随机化算法在T轮次中产生的K1校准误差最坏情况期望下界为${\Omega}(T^{0.528})$,这为MSR优化设置了障碍。为突破该障碍,已有研究通过外部遗憾(Kleinberg et al., 2023)以及基于下游任务动作数多项式依赖的遗憾界(Noarov et al., 2023;Roth and Shi, 2024)提出了若干MSR松弛方法。我们证明无需任何松弛即可突破该障碍:提出了一个高效随机化预测算法,保证期望MSR为$O(T\log T)$。此外,通过将MSR视为决策理论校准误差指标,我们探讨了校准的经济效用,并研究了其与现有指标的关系。