Frontier AI systems, including large language models and emerging agentic AI tools, offer significant operational benefits but present unique challenges to critical infrastructure (CI) environments due to their non-deterministic and emergent properties. While formal adoption is inherently cautious and tightly controlled due to strict regulatory oversight, widespread accessibility has catalysed shadow AI: the unsanctioned use of frontier AI outside established organisational controls. In CI settings, shadow AI bypasses established assurance and oversight mechanisms, amplifying risks to data protection, decision reliability, and regulatory compliance, with potential consequences for essential service delivery. We present the first empirical study of shadow AI in CI environments, characterising it as a systemic socio-technical condition of assurance erosion. Drawing on semi-structured interviews with senior executives and functional leaders across 27 Australian CI organisations (Communications, Energy, and Water and Sewerage sectors), we analyse how shadow AI manifests in practice, how it interacts with existing technical and governance controls, and the resulting security, assurance, and compliance risks. We develop an empirically derived threat model identifying three primary mechanisms of security degradation: (i) boundary bypass, where data flows circumvent established perimeters; (ii) unassessed capability expansion, where embedded AI features introduce latent risks; and (iii) loss of observability via governance circumvention, undermining forensic auditability and least-privilege enforcement. Our findings demonstrate that shadow AI introduces unmanaged risks that fundamentally challenge existing security and compliance frameworks, necessitating tailored, pathway-aligned governance and control strategies.

翻译：前沿人工智能系统，包括大型语言模型和新兴的智能体AI工具，提供了显著的运营效益，但由于其非确定性和涌现性，给关键基础设施环境带来了独特挑战。尽管由于严格的监管监督，正式采用过程本质上是谨慎且受到严格控制的，但广泛的可访问性催生了影子人工智能：即在既定组织控制之外未经授权使用前沿AI。在关键基础设施环境中，影子人工智能绕过了既定的保障和监督机制，加剧了数据保护、决策可靠性和监管合规性方面的风险，并可能对基本服务交付产生影响。我们提出了首个关于关键基础设施环境中影子人工智能的实证研究，将其描述为一种保障侵蚀的系统性社会技术条件。基于对澳大利亚27家关键基础设施组织（涵盖通信、能源及水务与污水处理行业）的高级管理人员和职能领导者的半结构化访谈，我们分析了影子人工智能在实践中的表现形式、其与现有技术和治理控制的互动方式，以及由此产生的安全、保障和合规风险。我们建立了一个经验推导的威胁模型，识别出三种主要的安全退化机制：（i）边界绕过，即数据流规避既定边界；（ii）未经评估的能力扩展，即嵌入式AI功能引入潜在风险；以及（iii）通过治理规避失去可观测性，破坏法证审计能力和最小权限原则的执行。我们的研究结果表明，影子人工智能引入了不受管理的风险，从根本上挑战了现有的安全和合规框架，因而需要针对性的、路径对齐的治理和控制策略。