It is widely agreed that when AI models assist decision-makers in high-stakes domains by predicting an outcome of interest, they should communicate the confidence of their predictions. However, empirical evidence suggests that decision-makers often struggle to determine when to trust a prediction based solely on this communicated confidence. In this context, recent theoretical and empirical work suggests a positive correlation between the utility of AI-assisted decision-making and the degree of alignment between the AI confidence and the decision-makers' confidence in their own predictions. Crucially, these findings do not yet elucidate the extent to which this alignment influences the complexity of learning to make optimal decisions through repeated interactions. In this paper, we address this question in the canonical case of binary predictions and binary decisions. We first show that this problem is equivalent to a two-armed online contextual learning problem with full feedback, and establish a lower bound of $Ω(\sqrt{|H| \cdot |B| \cdot T} )$ on the expected regret any learner can attain, where $H$ and $B$ denote the sets of human and AI confidence values. We then demonstrate that, under perfect alignment between AI and human confidence, a learner can attain an expected regret of $O(\sqrt{|H| \cdot T\log T})$ and, when $\sqrt{|H|} = O(\log T)$ and $B$ is countable, a non-trivial generalization of the Dvoretzky-Kiefer-Wolfowitz inequality improves the regret bound to $O(\sqrt{T\log T})$. Taken together, these results reveal that alignment can reduce the complexity of learning to make decisions with AI assistance. Experiments on real data from two different human-subject studies where participants solve simple decision-making tasks assisted by AI models show that our theoretical results are robust to violations of perfect alignment.

翻译：摘要：学界普遍认为，当AI模型通过预测感兴趣的结果来协助决策者处理高风险领域时，应传达其预测的置信度。然而，实证证据表明，决策者往往难以仅凭这种传达的置信度来判断何时应信任预测。在此背景下，近期理论与实证研究显示，AI辅助决策的效用与AI置信度及决策者自身预测置信度之间的对齐程度呈正相关。关键的是，这些发现尚未阐明这种对齐程度如何影响通过重复交互学习最优决策的复杂度。本文针对二元预测与二元决策的典型情形探讨该问题。我们首先证明该问题等价于一个具有完全反馈的双臂在线上下文学习问题，并建立了任何学习器可实现的期望遗憾下界$Ω(\sqrt{|H| \cdot |B| \cdot T} )$，其中$H$和$B$分别表示人类与AI的置信度取值集合。进一步研究表明，当AI与人类置信度完美对齐时，学习器可实现$O(\sqrt{|H| \cdot T\log T})$的期望遗憾；当$\sqrt{|H|} = O(\log T)$且$B$可数时，通过Dvoretzky-Kiefer-Wolfowitz不等式的非平凡推广，遗憾界可优化至$O(\sqrt{T\log T})$。综合而言，这些结果揭示了对齐能降低学习借助AI进行决策的复杂度。基于两项不同人类受试者实验中参与者借助AI模型完成简单决策任务的实际数据，我们验证了理论结果对完美对齐条件偏离具有鲁棒性。