This paper presents Bayesian techniques for conservative claims about software reliability, particularly when evidence suggests the software's executions are not statistically independent. We formalise informal notions of "doubting" that the executions are independent, and incorporate such doubts into reliability assessments. We develop techniques that reveal the extent to which independence assumptions can undermine conservatism in assessments, and identify conditions under which this impact is not significant. These techniques - novel extensions of conservative Bayesian inference (CBI) approaches - give conservative confidence bounds on the software's failure probability per execution. With illustrations in two application areas - nuclear power-plant safety and autonomous vehicle (AV) safety - our analyses reveals: 1) the confidence an assessor should possess before subjecting a system to operational testing. Otherwise, such testing is futile - favourable operational testing evidence will eventually decrease one's confidence in the system being sufficiently reliable; 2) the independence assumption supports conservative claims sometimes; 3) in some scenarios, observing a system operate without failure gives less confidence in the system than if some failures had been observed; 4) building confidence in a system is very sensitive to failures - each additional failure means significantly more operational testing is required, in order to support a reliability claim.
翻译:本文提出了用于对软件可靠性进行保守声明的贝叶斯技术,特别是当证据表明软件的执行并非统计独立时。我们形式化了关于“怀疑”执行非独立性的非正式概念,并将这种怀疑纳入可靠性评估中。我们开发了能够揭示独立性假设在多大程度上削弱评估保守性的技术,并确定了该影响不显著的条件。这些技术——保守贝叶斯推断方法的新颖扩展——给出了每次执行时软件失效概率的保守置信界限。通过在两个应用领域——核电站安全与自动驾驶车辆安全中的示例说明,我们的分析揭示了:1)评估者在将系统投入运行测试前应具备的置信度。否则,这种测试将徒劳无功——有利的运行测试证据最终会降低对系统足够可靠的置信度;2)独立性假设有时支持保守声明;3)在某些场景下,观察系统无故障运行比观察到某些故障时对系统的置信度更低;4)建立对系统的置信度对故障非常敏感——每增加一个故障就意味着需要显著更多的运行测试来支持可靠性声明。