In AI-assisted decision-making, humans often passively review AI's suggestion and decide whether to accept or reject it as a whole. In such a paradigm, humans are found to rarely trigger analytical thinking and face difficulties in communicating the nuances of conflicting opinions to the AI when disagreements occur. To tackle this challenge, we propose Human-AI Deliberation, a novel framework to promote human reflection and discussion on conflicting human-AI opinions in decision-making. Based on theories in human deliberation, this framework engages humans and AI in dimension-level opinion elicitation, deliberative discussion, and decision updates. To empower AI with deliberative capabilities, we designed Deliberative AI, which leverages large language models (LLMs) as a bridge between humans and domain-specific models to enable flexible conversational interactions and faithful information provision. An exploratory evaluation on a graduate admissions task shows that Deliberative AI outperforms conventional explainable AI (XAI) assistants in improving humans' appropriate reliance and task performance. Based on a mixed-methods analysis of participant behavior, perception, user experience, and open-ended feedback, we draw implications for future AI-assisted decision tool design.

翻译：在AI辅助决策中，人类通常被动地审阅AI的建议，并决定整体接受或拒绝。在此范式下，研究发现人类很少触发分析性思维，且在出现意见冲突时，难以向AI传达对立的细微差异。为应对这一挑战，我们提出"人机协商"这一创新框架，旨在促进人类在决策过程中对矛盾的人机意见进行反思与讨论。基于人类协商理论，该框架引导人类与AI开展维度级意见获取、协商性讨论及决策更新。为赋予AI协商能力，我们设计了协商型AI，利用大语言模型作为人类与领域特定模型之间的桥梁，实现灵活对话交互与可靠信息提供。在研究生录取任务上的探索性评估表明，相较于传统可解释AI助手，协商型AI在提升人类恰当信任与任务绩效方面表现更优。通过对参与者行为、感知、用户体验及开放式反馈的混合方法分析，我们为未来AI辅助决策工具的设计提出了启示。