Large Language Models (LLMs) are attracting significant research attention due to their instruction-following abilities, allowing users and developers to leverage LLMs for a variety of tasks. However, LLMs are vulnerable to prompt-injection attacks: a class of attacks that hijack the model's instruction-following abilities, changing responses to prompts to undesired, possibly malicious ones. In this work, we introduce Jatmo, a method for generating task-specific models resilient to prompt-injection attacks. Jatmo leverages the fact that LLMs can only follow instructions once they have undergone instruction tuning. It harnesses a teacher instruction-tuned model to generate a task-specific dataset, which is then used to fine-tune a base model (i.e., a non-instruction-tuned model). Jatmo only needs a task prompt and a dataset of inputs for the task: it uses the teacher model to generate outputs. For situations with no pre-existing datasets, Jatmo can use a single example, or in some cases none at all, to produce a fully synthetic dataset. Our experiments on seven tasks show that Jatmo models provide similar quality of outputs on their specific task as standard LLMs, while being resilient to prompt injections. The best attacks succeeded in less than 0.5% of cases against our models, versus 87% success rate against GPT-3.5-Turbo. We release Jatmo at https://github.com/wagner-group/prompt-injection-defense.

翻译：大型语言模型（LLMs）因其遵循指令的能力而备受研究关注，使用户和开发者能够利用LLMs执行多种任务。然而，LLMs易受提示注入攻击：这类攻击劫持模型的指令遵循能力，将响应引导至非预期的、可能恶意的内容。本文提出Jatmo，一种生成对提示注入攻击具有抵抗力的任务特定模型的方法。Jatmo利用LLMs仅在经过指令微调后才能遵循指令的特性，通过教师指令微调模型生成任务特定数据集，并用该数据集微调基础模型（即未经指令微调的模型）。Jatmo仅需任务提示和任务输入数据集：它使用教师模型生成输出。对于没有预设数据集的情况，Jatmo可利用单个示例（甚至无需示例）生成完全合成数据集。我们在七项任务上的实验表明，Jatmo模型在特定任务上的输出质量与标准LLMs相当，同时对提示注入攻击具有抵抗力。最有效的攻击在我们模型上的成功率低于0.5%，而对GPT-3.5-Turbo的攻击成功率达到87%。我们在https://github.com/wagner-group/prompt-injection-defense上发布Jatmo。