The integration of Artificial Intelligence (AI) into safety-critical systems introduces a new reliability paradigm: silent failures, where AI produces confident but incorrect outputs that can be dangerous. This paper introduces the Formal Assurance and Monitoring Environment (FAME), a novel framework that confronts this challenge. FAME synergizes the mathematical rigor of offline formal synthesis with the vigilance of online runtime monitoring to create a verifiable safety net around opaque AI components. We demonstrate its efficacy in an autonomous vehicle perception system, where FAME successfully detected 93.5% of critical safety violations that were otherwise silent. By contextualizing our framework within the ISO 26262 and ISO/PAS 8800 standards, we provide reliability engineers with a practical, certifiable pathway for deploying trustworthy AI. FAME represents a crucial shift from accepting probabilistic performance to enforcing provable safety in next-generation systems.

翻译：人工智能（AI）在安全关键系统中的集成引入了一种新的可靠性范式：静默失效，即AI产生自信但错误的输出，这可能带来危险。本文介绍了形式化保证与监控环境（FAME），这是一个应对该挑战的新型框架。FAME将离线形式化合成的数学严谨性与在线运行时监控的警觉性相结合，为不透明的AI组件构建了一个可验证的安全网。我们在自动驾驶汽车感知系统中验证了其有效性，FAME成功检测到了93.5%原本静默的关键安全违规。通过将我们的框架置于ISO 26262和ISO/PAS 8800标准背景下，我们为可靠性工程师提供了一条实用、可认证的部署可信AI的路径。FAME代表了从接受概率性能到在下一代系统中强制执行可证明安全性的关键转变。