Human oversight is currently discussed as a potential safeguard to counter some of the negative aspects of high-risk AI applications. This prompts a critical examination of the role and conditions necessary for what is prominently termed effective or meaningful human oversight of these systems. This paper investigates effective human oversight by synthesizing insights from psychological, legal, philosophical, and technical domains. Based on the claim that the main objective of human oversight is risk mitigation, we propose a viable understanding of effectiveness in human oversight: for human oversight to be effective, the oversight person has to have (a) sufficient causal power with regard to the system and its effects, (b) suitable epistemic access to relevant aspects of the situation, (c) self-control, and (d) fitting intentions for their role. Furthermore, we argue that this is equivalent to saying that an oversight person is effective if and only if they are morally responsible and have fitting intentions. Against this backdrop, we suggest facilitators and inhibitors of effectiveness in human oversight when striving for practical applicability. We discuss factors in three domains, namely, the technical design of the system, individual factors of oversight persons, and the environmental circumstances in which they operate. Finally, this paper scrutinizes the upcoming AI Act of the European Union -- in particular Article 14 on Human Oversight -- as an exemplary regulatory framework in which we study the practicality of our understanding of effective human oversight. By analyzing the provisions and implications of the European AI Act proposal, we pinpoint how far that proposal aligns with our analyses regarding effective human oversight as well as how it might get enriched by our conceptual understanding of effectiveness in human oversight.
翻译:人类监督目前被探讨为应对高风险人工智能应用负面影响的潜在保障措施。这促使我们严格审视所谓有效或有意义的人类监督系统所需的条件与角色定位。本研究通过整合心理学、法学、哲学和技术领域的洞见,系统探究人类监督的有效性问题。基于人类监督的主要目标是风险缓解这一主张,我们提出了人类监督有效性的可行理解框架:要实现有效监督,监督者必须同时具备(a)对系统及其影响的充分因果控制力,(b)对情境相关要素的合适认知通道,(c)自我控制能力,以及(d)符合其角色定位的恰当意图。进一步论证表明,这等同于判定监督者有效的充要条件是其具备道德责任能力与恰当意图。在此基准上,我们提出了实现实践应用时人类监督有效性的促进因素与阻碍因素。我们从三个领域展开讨论:系统的技术设计、监督者的个体因素以及运作环境条件。最后,本文以欧盟即将出台的《人工智能法案》——特别是第14条关于人类监督的条款——作为典型案例,研究我们对有效人类监督的理解框架在监管体系中的适用性。通过分析欧盟人工智能法案提案的条款与影响,我们既揭示了该提案与我们对有效人类监督分析的一致性程度,也指出了我们关于人类监督有效性的概念性理解如何能进一步丰富该监管框架。