AI and ML models have already found many applications in critical domains, such as healthcare and criminal justice. However, fully automating such high-stakes applications can raise ethical or fairness concerns. Instead, in such cases, humans should be assisted by automated systems so that the two parties reach a joint decision, stemming out of their interaction. In this work we conduct an empirical study to identify how uncertainty estimates and model explanations affect users' reliance, understanding, and trust towards a model, looking for potential benefits of bringing the two together. Moreover, we seek to assess how users' behaviour is affected by their own self-confidence in their abilities to perform a certain task, while we also discuss how the latter may distort the outcome of an analysis based on agreement and switching percentages.
翻译:人工智能和机器学习模型已在医疗、刑事司法等关键领域得到广泛应用。然而,完全自动化此类高风险应用可能引发伦理或公平性问题。相反,在这些情况下,人类应借助自动化系统辅助,通过双方互动达成联合决策。本研究通过实证分析,探究不确定性估计与模型解释如何影响用户对模型的依赖、理解和信任,并寻找二者结合可能带来的潜在优势。此外,我们试图评估用户对自身执行特定任务能力的自信心如何影响其行为,同时讨论后者如何可能扭曲基于一致性和切换百分比的分析结果。