Despite the growing prevalence of human-AI decision making, the human-AI team's decision performance often remains suboptimal, partially due to insufficient examination of humans' own reasoning. In this paper, we explore designing AI systems that directly analyze humans' decision rationales and encourage critical reflection of their own decisions. We introduce the AI-Assisted Critical Thinking (AACT) framework, which leverages a domain-specific AI model's counterfactual analysis of human decision to help decision-makers identify potential flaws in their decision argument and support the correction of them. Through a case study on house price prediction, we find that AACT outperforms traditional AI-based decision-support in reducing over-reliance on AI, though also triggering higher cognitive load. Subgroup analysis reveals AACT can be particularly beneficial for some decision-makers such as those very familiar with AI technologies. We conclude by discussing the practical implications of our findings, use cases and design choices of AACT, and considerations for using AI to facilitate critical thinking.

翻译：尽管人机协同决策日益普遍，但人机团队的决策表现往往仍不理想，部分原因在于对人类自身推理过程的审视不足。本文探讨如何设计能够直接分析人类决策依据并鼓励对其决策进行批判性反思的AI系统。我们提出AI辅助批判性思维框架，该框架利用领域专用AI模型对人类决策的反事实分析，帮助决策者识别其决策论证中的潜在缺陷，并支持修正这些缺陷。通过房价预测的案例研究，我们发现AACT在降低对AI的过度依赖方面优于传统的基于AI的决策支持系统，但也会引发更高的认知负荷。亚组分析表明，AACT对某些决策者（如非常熟悉AI技术的群体）可能特别有益。最后，我们讨论了研究结果的实际意义、AACT的适用场景与设计选择，以及利用AI促进批判性思维的注意事项。