CONTEXT: Assurance Cases (ACs) are prepared to argue that the system's desired quality attributes (e.g., safety or security) are satisfied. While there is strong adoption of ACs, practitioners are often left asking an important question: are we confident that the claims made by the case are true? While many confidence assessment methods (CAMs) exist, little is known about the use of these methods in practice OBJECTIVE: Develop an understanding of the current state of practice for AC confidence assessment: what methods are used in practice and what barriers exist for their use? METHOD: Structured interviews were performed with practitioners with experience contributing to real-world ACs. Open-coding was performed on transcripts. A description of the current state of AC practice and future considerations for researchers was synthesized from the results. RESULTS: A total of n = 19 practitioners were interviewed. The most common CAMs were (peer-)review of ACs, dialectic reasoning ("defeaters"), and comparing against checklists. Participants preferred qualitative methods and expressed concerns about quantitative CAMs. Barriers to using CAMs included additional work, inadequate guidance, subjectivity and interpretation of results, and trustworthiness of methods. CONCLUSION: While many CAMs are described in the literature there is a gap between the proposed methods and needs of practitioners. Researchers working in this area should consider the need to: connect CAMs to established practices, use CAMs to communicate with interest holders, crystallize the details of CAM application, curate accessible guidance, and confirm that methods are trustworthy.
翻译:背景:保证案例(ACs)旨在论证系统所需的质量属性(如安全性或保密性)已得到满足。尽管ACs已被广泛采用,从业者仍常面临一个重要问题:我们是否有信心确信案例所提出的主张是真实的?虽然存在多种置信度评估方法(CAMs),但关于这些方法在实际中的应用情况却知之甚少。目标:理解当前AC置信度评估的实践现状:实践中使用了哪些方法,以及应用这些方法存在哪些障碍?方法:对具有真实世界AC贡献经验的从业者进行了结构化访谈。对访谈转录文本进行了开放式编码。根据结果综合描述了当前AC实践的现状,并为研究者提出了未来考量方向。结果:共计访谈了n = 19位从业者。最常见的CAMs包括对AC的(同行)评审、辩证推理(“反驳项”)以及对照检查清单进行比较。参与者更倾向于定性方法,并对定量CAMs表示担忧。使用CAMs的障碍包括额外的工作量、指导不足、结果的主观性与解释问题,以及方法的可信度。结论:尽管文献中描述了许多CAMs,但所提出的方法与从业者的需求之间存在差距。该领域的研究者应考虑以下需求:将CAMs与既定实践相结合,利用CAMs与利益相关方沟通,明确CAM应用的具体细节,整理易于获取的指导材料,并确认方法的可信度。