A major concern amongst AI safety practitioners is the possibility of loss of control, whereby humans lose the ability to exert control over increasingly advanced AI systems. The range of concerns is wide, spanning current day risks to future existential risks, and a range of loss of control pathways from rapid AI self-exfiltration scenarios to more gradual disempowerment scenarios. In this work we set out to firstly, provide a more structured framework for discussing and characterizing loss of control and secondly, to use this framework to assist those responsible for the safe operation of AI-containing socio-technical systems to identify causal factors leading to loss of control. We explore how these two needs can be better met by making use of a methodology developed within the safety-critical systems community known as STAMP and its associated hazard analysis technique of STPA. We select the STAMP methodology primarily because it is based around a world-view that socio-technical systems can be functionally modeled as control structures, and that safety issues arise when there is a loss of control in these structures.

翻译：人工智能安全从业者的一个主要担忧是失控的可能性，即人类逐渐丧失对日益先进的人工智能系统施加控制的能力。相关担忧的范围很广，既涵盖当前风险，也涉及未来存在的风险，其失控路径既包括AI快速自我渗透的突发情境，也包含权力逐渐被剥夺的渐进情境。本研究的首要目标是提供一个更结构化的框架，用于讨论和刻画失控现象；其次，旨在利用该框架协助负责含AI的社会技术系统安全运行的人员识别导致失控的因果因素。我们探讨如何通过采用安全关键系统领域开发的方法论——STAMP及其相关的危险分析技术STPA——来更好地满足这两方面需求。选择STAMP方法论的主要依据在于：其世界观认为社会技术系统可在功能上建模为控制结构，而安全问题正源于这些结构中控制力的丧失。