Large language models (LLMs) extended as systems, such as ChatGPT, have begun supporting third-party applications. These LLM apps leverage the de facto natural language-based automated execution paradigm of LLMs: that is, apps and their interactions are defined in natural language, provided access to user data, and allowed to freely interact with each other and the system. These LLM app ecosystems resemble the settings of earlier computing platforms, where there was insufficient isolation between apps and the system. Because third-party apps may not be trustworthy, and exacerbated by the imprecision of the natural language interfaces, the current designs pose security and privacy risks for users. In this paper, we propose SecGPT, an architecture for LLM-based systems that aims to mitigate the security and privacy issues that arise with the execution of third-party apps. SecGPT's key idea is to isolate the execution of apps and more precisely mediate their interactions outside of their isolated environments. We evaluate SecGPT against a number of case study attacks and demonstrate that it protects against many security, privacy, and safety issues that exist in non-isolated LLM-based systems. The performance overhead incurred by SecGPT to improve security is under 0.3x for three-quarters of the tested queries. To foster follow-up research, we release SecGPT's source code at https://github.com/llm-platform-security/SecGPT.

翻译：大型语言模型（LLMs）扩展为系统（如ChatGPT）后，开始支持第三方应用。这些LLM应用利用了基于自然语言的自动化执行范式——即应用及其交互通过自然语言定义、可访问用户数据，并允许彼此及与系统自由交互。此类LLM应用生态与早期计算平台的设置类似，应用与系统之间缺乏充分隔离。由于第三方应用可能不可信，且自然语言接口存在不精确性问题，当前设计给用户带来了安全与隐私风险。本文提出SecGPT，一种面向基于LLM系统的架构，旨在缓解第三方应用执行中产生的安全与隐私问题。SecGPT的核心思想是隔离应用执行，并更精确地协调其在隔离环境之外的交互。我们通过多个案例攻击对SecGPT进行评估，证明其能防范非隔离型LLM系统中的诸多安全、隐私及可靠性问题。为提升安全性，SecGPT在四分之三的测试查询中引入的性能开销低于0.3倍。为促进后续研究，我们在https://github.com/llm-platform-security/SecGPT 开源了SecGPT的源代码。