Artificial intelligence (AI) is broadly deployed as an advisor to human decision-makers: AI recommends a decision and a human accepts or rejects the advice. This approach, however, has several limitations: People frequently ignore accurate advice and rely too much on inaccurate advice, and their decision-making skills may deteriorate over time. Here, we compare the AI-as-advisor approach to the hybrid confirmation tree (HCT), an alternative strategy that preserves the independence of human and AI judgments. The HCT elicits a human judgment and an AI judgment independently of each other. If they agree, that decision is accepted. If not, a second human breaks the tie. For the comparison, we used 10 datasets from various domains, including medical diagnostics and misinformation discernment, and a subset of four datasets in which AI also explained its decision. The HCT outperformed the AI-as-advisor approach in all datasets. The HCT also performed better in almost all cases in which AI offered an explanation of its judgment. Using signal detection theory to interpret these results, we find that the HCT outperforms the AI-as-advisor approach because people cannot discriminate well enough between correct and incorrect AI advice. Overall, the HCT is a robust, accurate, and transparent alternative to the AI-as-advisor approach, offering a simple mechanism to tap into the wisdom of hybrid crowds.
翻译:人工智能(AI)被广泛部署为人类决策者的顾问:AI推荐决策方案,人类选择接受或拒绝该建议。然而,这种方法存在若干局限:人们经常忽视准确的建议,过度依赖不准确建议,且其决策技能可能随时间退化。本研究将AI作为顾问的方法与混合确认树(HCT)进行对比,后者是一种保持人类与AI判断独立性的替代策略。HCT在互不干扰的情况下分别获取人类判断与AI判断:若两者一致则采纳该决策;若不一致则由第二位人类决策者打破平局。我们采用涵盖医学诊断、虚假信息识别等领域的10个数据集进行对比,其中4个数据集包含AI对其决策的解释。结果显示,HCT在所有数据集中均优于AI作为顾问的方法,在AI提供判断解释的案例中几乎全部表现更佳。通过信号检测理论解读结果发现,HCT之所以优于AI作为顾问的方法,是因为人类无法有效区分AI建议的正确与否。总体而言,HCT作为AI顾问范式的稳健、准确且透明的替代方案,为挖掘混合群体智慧提供了简洁的机制。