Prior work has identified a resilient phenomenon that threatens the performance of human-AI decision-making teams: overreliance, when people agree with an AI, even when it is incorrect. Surprisingly, overreliance does not reduce when the AI produces explanations for its predictions, compared to only providing predictions. Some have argued that overreliance results from cognitive biases or uncalibrated trust, attributing overreliance to an inevitability of human cognition. By contrast, our paper argues that people strategically choose whether or not to engage with an AI explanation, demonstrating empirically that there are scenarios where AI explanations reduce overreliance. To achieve this, we formalize this strategic choice in a cost-benefit framework, where the costs and benefits of engaging with the task are weighed against the costs and benefits of relying on the AI. We manipulate the costs and benefits in a maze task, where participants collaborate with a simulated AI to find the exit of a maze. Through 5 studies (N = 731), we find that costs such as task difficulty (Study 1), explanation difficulty (Study 2, 3), and benefits such as monetary compensation (Study 4) affect overreliance. Finally, Study 5 adapts the Cognitive Effort Discounting paradigm to quantify the utility of different explanations, providing further support for our framework. Our results suggest that some of the null effects found in literature could be due in part to the explanation not sufficiently reducing the costs of verifying the AI's prediction.
翻译:已有研究识别出一种影响人机协同决策团队绩效的顽固现象:过度依赖,即人们即使AI判断错误时仍然认同其意见。令人惊讶的是,相较于仅提供预测结果,当AI为其预测提供解释时,过度依赖现象并未减少。部分学者认为过度依赖源于认知偏差或校准不当的信任,将其归因于人类认知的必然性。与之相反,本文论证人们会策略性地选择是否参与AI解释,并通过实证表明在某些情境下AI解释能够减少过度依赖。为此,我们在成本-收益框架下正式化这一策略选择:权衡参与任务本身的成本收益与依赖AI的成本收益。我们通过迷宫任务操纵成本收益变量,让参与者与模拟AI协作寻找迷宫出口。基于5项研究(N=731)发现:任务难度(研究1)、解释难度(研究2、3)等成本因素,以及金钱报酬(研究4)等收益因素均会影响过度依赖。最后,研究5采用认知努力贴现范式量化不同解释的效用,进一步支撑我们的理论框架。研究结果表明,已有文献中部分无效发现可能源于解释未能充分降低验证AI预测的成本。