The use of artificial intelligence (AI) in working environments with individuals, known as Human-AI Collaboration (HAIC), has become essential in a variety of domains, boosting decision-making, efficiency, and innovation. Despite HAIC's wide potential, evaluating its effectiveness remains challenging due to the complex interaction of components involved. This paper provides a detailed analysis of existing HAIC evaluation approaches and develops a fresh paradigm for more effectively evaluating these systems. Our framework includes a structured decision tree which assists to select relevant metrics based on distinct HAIC modes (AI-Centric, Human-Centric, and Symbiotic). By including both quantitative and qualitative metrics, the framework seeks to represent HAIC's dynamic and reciprocal nature, enabling the assessment of its impact and success. This framework's practicality can be examined by its application in an array of domains, including manufacturing, healthcare, finance, and education, each of which has unique challenges and requirements. Our hope is that this study will facilitate further research on the systematic evaluation of HAIC in real-world applications.

翻译：人工智能（AI）在与个体协作的工作环境中的应用，即人机协作（HAIC），已在多个领域变得至关重要，提升了决策质量、工作效率与创新能力。尽管HAIC具有广泛潜力，但由于涉及组件的复杂交互性，评估其有效性仍具挑战性。本文对现有HAIC评估方法进行了详细分析，并构建了一种用于更有效评估此类系统的新范式。我们的框架包含一个结构化决策树，可依据不同HAIC模式（以AI为中心、以人为中心、共生协作）协助选择相关指标。通过纳入定量与定性指标，该框架力图体现HAIC动态且互惠的本质，从而实现对其实施效果与成功度的评估。本框架的实用性可通过其在制造、医疗、金融、教育等领域的应用得到验证，每个领域皆具备独特的挑战与需求。我们希望本研究能推动针对实际应用中HAIC系统化评估的进一步探索。