Large Language Models (LLMs) are attracting significant research attention due to their instruction-following abilities, allowing users and developers to leverage LLMs for a variety of tasks. However, LLMs are vulnerable to prompt-injection attacks: a class of attacks that hijack the model's instruction-following abilities, changing responses to prompts to undesired, possibly malicious ones. In this work, we introduce Jatmo, a method for generating task-specific models resilient to prompt-injection attacks. Jatmo leverages the fact that LLMs can only follow instructions once they have undergone instruction tuning. It harnesses a teacher instruction-tuned model to generate a task-specific dataset, which is then used to fine-tune a base model (i.e., a non-instruction-tuned model). Jatmo only needs a task prompt and a dataset of inputs for the task: it uses the teacher model to generate outputs. For situations with no pre-existing datasets, Jatmo can use a single example, or in some cases none at all, to produce a fully synthetic dataset. Our experiments on six tasks show that Jatmo models provide the same quality of outputs on their specific task as standard LLMs, while being resilient to prompt injections. The best attacks succeeded in less than 0.5% of cases against our models, versus over 90% success rate against GPT-3.5-Turbo. We release Jatmo at https://github.com/wagner-group/prompt-injection-defense.

翻译：大型语言模型（LLMs）因其指令遵循能力而备受关注，使用户和开发者能够将其应用于多种任务。然而，LLMs容易受到提示注入攻击——这类攻击劫持模型的指令遵循能力，将响应结果篡改为预期外甚至恶意的内容。本文提出Jatmo方法，用于生成对提示注入攻击具有鲁棒性的任务特定模型。Jatmo利用LLMs只有经过指令微调后才能遵循指令这一特性，通过教师指令微调模型生成任务特定数据集，进而对基础模型（即非指令微调模型）进行微调。该方法仅需任务提示和任务输入数据集，借助教师模型生成输出。对于无现有数据集的情形，Jatmo可使用单一示例（某些情况下甚至不需要示例）生成全合成数据集。我们在六项任务上的实验表明，Jatmo模型在特定任务上保持与标准LLMs相同的输出质量，同时具备提示注入防御能力。针对我们的模型，最优攻击的成功率低于0.5%，而对GPT-3.5-Turbo的攻击成功率超过90%。Jatmo开源代码发布在https://github.com/wagner-group/prompt-injection-defense。