The use of partially automated driving systems raises concerns about potential responsibility issues, posing risk to the system safety, acceptance, and adoption of these technologies. The concept of meaningful human control has emerged in response to the responsibility gap problem, requiring the fulfillment of two conditions, tracking and tracing. While this concept has provided important philosophical and design insights on automated driving systems, there is currently little knowledge on how meaningful human control relates to subjective experiences of actual users of these systems. To address this gap, our study aimed to investigate the alignment between the degree of meaningful human control and drivers' perceptions of safety and trust in a real-world partially automated driving system. We utilized previously collected data from interviews with Tesla "Full Self-Driving" (FSD) Beta users, investigating the alignment between the user perception and how well the system was tracking the users' reasons. We found that tracking of users' reasons for driving tasks (such as safe maneuvers) correlated with perceived safety and trust, albeit with notable exceptions. Surprisingly, failure to track lane changing and braking reasons was not necessarily associated with negative perceptions of safety. However, the failure of the system to track expected maneuvers in dangerous situations always resulted in low trust and perceived lack of safety. Overall, our analyses highlight alignment points but also possible discrepancies between perceived safety and trust on the one hand, and meaningful human control on the other hand. Our results can help the developers of automated driving technology to design systems under meaningful human control and are perceived as safe and trustworthy.
翻译:部分自动驾驶系统的使用引发了对潜在责任问题的担忧,这给系统的安全性、接受度和技术应用带来了风险。有意义的人类控制概念应运而生,以应对责任鸿沟问题,其要求同时满足两个条件:跟踪与追溯。尽管该概念已为自动驾驶系统提供了重要的哲学与设计洞见,但目前对有意义的人类控制如何与实际系统用户的主观体验相关联知之甚少。为填补这一空白,本研究旨在探讨真实世界部分自动驾驶系统中,有意义的人类控制程度与驾驶员对安全性和信任感的感知之间的一致性。我们利用此前从特斯拉“完全自动驾驶”(FSD)Beta版用户访谈中收集的数据,探究用户感知与系统对用户意图的跟踪效果之间的匹配度。研究发现,系统对用户驾驶任务意图(如安全操作)的跟踪与感知安全性和信任度相关,但存在显著例外。令人惊讶的是,系统在变道和制动意图跟踪上的失败并不必然导致负面安全感知;然而,系统在危险情境下未能跟踪预期操作,始终引发低信任度和安全缺失感。总体而言,我们的分析既凸显了感知安全性与信任度同有意义的人类控制之间的契合点,也揭示了可能存在的差异。研究结果可帮助自动驾驶技术开发者在有意义的人类控制框架下设计系统,使其既安全又值得信赖。