In shared autonomy, a critical tension arises when an automated assistant must choose between obeying a human's instruction and deliberately overriding it to prevent harm. This safety-critical behavior is known as intelligent disobedience. To formalize this dynamic, this paper introduces the Intelligent Disobedience Game (IDG), a sequential game-theoretic framework based on Stackelberg games that models the interaction between a human leader and an assistive follower operating under asymmetric information. It characterizes optimal strategies for both agents across multi-step scenarios, identifying strategic phenomena such as ``safety traps,'' where the system indefinitely avoids harm but fails to achieve the human's goal. The IDG provides a needed mathematical foundation that enables both the algorithmic development of agents that can learn safe non-compliance and the empirical study of how humans perceive and trust disobedient AI. The paper further translates the IDG into a shared control Multi-Agent Markov Decision Process representation, forming a compact computational testbed for training reinforcement learning agents.

翻译：在共享自主系统中，当自动化助手面临服从人类指令与主动否决指令以避免伤害之间的抉择时，会产生一种关键张力。这种安全攸关的行为被称为"智能不服从"。为形式化这一动态过程，本文提出智能不服从博弈（IDG）——一种基于斯塔克尔伯格博弈的序列博弈理论框架，用于建模人类领导者与辅助跟随者在非对称信息条件下的交互行为。该框架刻画了多步场景下两类智能体的最优策略，识别出"安全陷阱"等策略性现象——系统虽能无限期避免伤害却无法实现人类目标。智能不服从博弈提供了必要的数学基础，既能支持可学习安全违抗行为的智能体算法开发，又能开展关于人类如何感知与信任不服从型人工智能的实证研究。本文进一步将智能不服从博弈转化为共享控制的多智能体马尔可夫决策过程表征，构建用于训练强化学习智能体的紧凑计算测试平台。