Large Language Model (LLM) systems are inherently compositional, with individual LLM serving as the core foundation with additional layers of objects such as plugins, sandbox, and so on. Along with the great potential, there are also increasing concerns over the security of such probabilistic intelligent systems. However, existing studies on LLM security often focus on individual LLM, but without examining the ecosystem through the lens of LLM systems with other objects (e.g., Frontend, Webtool, Sandbox, and so on). In this paper, we systematically analyze the security of LLM systems, instead of focusing on the individual LLMs. To do so, we build on top of the information flow and formulate the security of LLM systems as constraints on the alignment of the information flow within LLM and between LLM and other objects. Based on this construction and the unique probabilistic nature of LLM, the attack surface of the LLM system can be decomposed into three key components: (1) multi-layer security analysis, (2) analysis of the existence of constraints, and (3) analysis of the robustness of these constraints. To ground this new attack surface, we propose a multi-layer and multi-step approach and apply it to the state-of-art LLM system, OpenAI GPT4. Our investigation exposes several security issues, not just within the LLM model itself but also in its integration with other components. We found that although the OpenAI GPT4 has designed numerous safety constraints to improve its safety features, these safety constraints are still vulnerable to attackers. To further demonstrate the real-world threats of our discovered vulnerabilities, we construct an end-to-end attack where an adversary can illicitly acquire the user's chat history, all without the need to manipulate the user's input or gain direct access to OpenAI GPT4. Our demo is in the link: https://fzwark.github.io/LLM-System-Attack-Demo/

翻译：大语言模型系统本质上是组合式的，以单个LLM为核心基础，叠加插件、沙箱等附加对象层。在展现巨大潜力的同时，此类概率智能系统的安全性也日益引发关注。然而，现有研究多聚焦于单个LLM的安全性，并未从包含其他对象（如前段、网页工具、沙箱等）的LLM系统生态视角进行审视。本文不局限于单个LLM，而是系统性地分析LLM系统的安全性。为此，我们基于信息流理论，将LLM系统的安全性形式化为约束条件，确保LLM内部及LLM与其他对象间的信息流对齐。基于该框架及LLM独特的概率特性，LLM系统的攻击面可分解为三大关键组件：（1）多层安全分析，（2）约束存在性分析，（3）约束鲁棒性分析。为验证这一新型攻击面，我们提出多层多步方法，并将其应用于当前最先进的LLM系统OpenAI GPT4。研究发现不仅LLM模型自身存在安全问题，其与其他组件的集成过程同样存在安全隐患。尽管OpenAI GPT4设计了大量安全约束以提升防护能力，但这些约束仍易受攻击者利用。为展示所发现漏洞的真实威胁，我们构建了端到端攻击：攻击者无需操纵用户输入或直接访问OpenAI GPT4，即可非法获取用户聊天历史。演示链接：https://fzwark.github.io/LLM-System-Attack-Demo/