The use of AI technologies is percolating into the secure development of software-based systems, with an increasing trend of composing AI-based subsystems (with uncertain levels of performance) into automated pipelines. This presents a fundamental research challenge and poses a serious threat to safety-critical domains (e.g., aviation). Despite the existing knowledge about uncertainty in risk analysis, no previous work has estimated the uncertainty of AI-augmented systems given the propagation of errors in the pipeline. We provide the formal underpinnings for capturing uncertainty propagation, develop a simulator to quantify uncertainty, and evaluate the simulation of propagating errors with two case studies. We discuss the generalizability of our approach and present policy implications and recommendations for aviation. Future work includes extending the approach and investigating the required metrics for validation in the aviation domain.

翻译：人工智能技术正逐步渗透到基于软件的系统的安全开发中，将基于人工智能的子系统（具有不确定的性能水平）组合成自动化管道的趋势日益增长。这提出了一个基础性的研究挑战，并对安全关键领域（如航空）构成了严重威胁。尽管风险分析领域已有关于不确定性的知识，但先前的研究尚未在考虑管道中误差传播的情况下，对人工智能增强系统的不确定性进行量化估计。我们为捕捉不确定性传播提供了形式化理论基础，开发了一个用于量化不确定性的模拟器，并通过两个案例研究评估了误差传播的模拟过程。我们讨论了所提方法的普适性，并提出了针对航空领域的政策影响与建议。未来的工作包括扩展该方法，并研究航空领域验证所需的具体度量指标。