As AI assistants become integrated into safety engineering workflows for Physical AI systems, a critical question emerges: does AI assistance improve safety analysis quality, or introduce systematic blind spots that surface only through post-deployment incidents? This paper develops a formal framework for AI assistance in safety analysis. We first establish why safety engineering resists benchmark-driven evaluation: safety competence is irreducibly multidimensional, constrained by context-dependent correctness, inherent incompleteness, and legitimate expert disagreement. We formalize this through a five-dimensional competence framework capturing domain knowledge, standards expertise, operational experience, contextual understanding, and judgment. We introduce the competence shadow: the systematic narrowing of human reasoning induced by AI-generated safety analysis. The shadow is not what the AI presents, but what it prevents from being considered. We formalize four canonical human-AI collaboration structures and derive closed-form performance bounds, demonstrating that the competence shadow compounds multiplicatively to produce degradation far exceeding naive additive estimates. The central finding is that AI assistance in safety engineering is a collaboration design problem, not a software procurement decision. The same tool degrades or improves analysis quality depending entirely on how it is used. We derive non-degradation conditions for shadow-resistant workflows and call for a shift from tool qualification toward workflow qualification for trustworthy Physical AI.

翻译：随着AI辅助工具融入物理AI系统的安全工程工作流，一个关键问题浮现：AI辅助究竟是提升了安全分析质量，还是引入了系统性的盲区，直至系统部署后发生事故才暴露出来？本文建立了一个形式化框架，用于分析安全工程中的AI辅助。我们首先论证了为何安全工程难以通过基准测试评估：安全能力具有不可约的多维性，受限于上下文依赖的正确性、固有的不完整性以及专家间的合理分歧。我们通过构建一个五维能力框架（涵盖领域知识、标准专长、操作经验、情境理解与判断力）对此进行了形式化。我们引入“能力暗影”概念：即由AI生成的安全分析所导致的人类推理系统性收窄。暗影并非AI呈现的内容，而是它阻止人们考虑的内容。我们对四种典型的人机协作结构进行了形式化建模，并推导出闭式性能界限，证明能力暗影会以乘法效应叠加，产生远超朴素加法估计的性能退化。核心结论是：安全工程中的AI辅助是一个协作设计问题，而非软件采购决策。同一工具对分析质量是提升还是降低，完全取决于其使用方式。我们推导了抗暗影工作流的非退化条件，并呼吁在面向可信物理AI时，将关注点从工具认证转向工作流认证。