Threat modeling is a popular method to securely develop systems by achieving awareness of potential areas of future damage caused by adversaries. However, threat modeling for systems relying on Artificial Intelligence is still not well explored. While conventional threat modeling methods and tools did not address AI-related threats, research on this amalgamation still lacks solutions capable of guiding and automating the process, as well as providing evidence that the methods hold up in practice. Consequently, this paper presents ThreatFinderAI, an approach and tool providing guidance and automation to model AI-related assets, threats, countermeasures, and quantify residual risks. To evaluate the practicality of the approach, participants were tasked to recreate a threat model developed by cybersecurity experts of an AI-based healthcare platform. Secondly, the approach was used to identify and discuss strategic risks in an LLM-based application through a case study. Overall, the solution's usability was well-perceived and effectively supports threat identification and risk discussion.

翻译：威胁建模是一种通过识别潜在攻击者可能造成损害的区域来安全开发系统的常用方法。然而，对于依赖人工智能的系统的威胁建模仍未得到充分探索。传统威胁建模方法与工具未能解决人工智能相关威胁，而关于二者融合的研究仍缺乏能够指导并自动化该过程、同时提供方法在实践中有效证据的解决方案。为此，本文提出ThreatFinderAI——一种为人工智能相关资产、威胁、防护措施建模及量化残余风险提供指导与自动化的方法与工具。为评估该方法的实用性，研究要求参与者复现网络安全专家为某人工智能医疗平台开发的威胁模型。其次，通过案例研究将该方法应用于基于大语言模型的应用程序中，以识别并讨论其战略风险。总体而言，该解决方案的可用性获得积极评价，能有效支持威胁识别与风险研讨。