With the development of Human-AI Collaboration in Classification (HAI-CC), integrating users and AI predictions becomes challenging due to the complex decision-making process. This process has three options: 1) AI autonomously classifies, 2) learning to complement, where AI collaborates with users, and 3) learning to defer, where AI defers to users. Despite their interconnected nature, these options have been studied in isolation rather than as components of a unified system. In this paper, we address this weakness with the novel HAI-CC methodology, called Learning to Complement and to Defer to Multiple Users (LECODU). LECODU not only combines learning to complement and learning to defer strategies, but it also incorporates an estimation of the optimal number of users to engage in the decision process. The training of LECODU maximises classification accuracy and minimises collaboration costs associated with user involvement. Comprehensive evaluations across real-world and synthesized datasets demonstrate LECODU's superior performance compared to state-of-the-art HAI-CC methods. Remarkably, even when relying on unreliable users with high rates of label noise, LECODU exhibits significant improvement over both human decision-makers alone and AI alone.

翻译：随着人机协同分类（HAI-CC）的发展，由于决策过程的复杂性，整合用户与人工智能预测结果面临挑战。该决策过程包含三种选项：1）人工智能自主分类；2）学习互补，即人工智能与用户协同工作；3）学习递延，即人工智能将决策权递延给用户。尽管这些选项本质相互关联，现有研究多将其孤立探讨，而非作为统一系统的组成部分。本文针对这一不足，提出了一种新颖的HAI-CC方法，称为面向多用户的学习互补与学习递延（LECODU）。LECODU不仅融合了学习互补与学习递延策略，还引入了对决策过程中最优参与用户数量的估计。LECODU的训练目标在于最大化分类准确率，并最小化因用户参与产生的协同成本。通过在真实数据集与合成数据集上的综合评估，LECODU展现出优于当前最先进HAI-CC方法的性能。值得注意的是，即使在用户标签噪声率较高、可靠性不足的情况下，LECODU相较于纯人工决策或纯人工智能决策仍表现出显著提升。