Prompt injection attacks insert malicious instructions into an LLM's input to steer it toward an attacker-chosen task instead of the intended one. Existing detection defenses typically classify any input with instruction as malicious, leading to misclassification of benign inputs containing instructions that align with the intended task. In this work, we account for the instruction hierarchy and distinguish among three categories: inputs with misaligned instructions, inputs with aligned instructions, and non-instruction inputs. We introduce AlignSentinel, a three-class classifier that leverages features derived from LLM's attention maps to categorize inputs accordingly. To support evaluation, we construct the first systematic benchmark containing inputs from all three categories. Experiments on both our benchmark and existing ones--where inputs with aligned instructions are largely absent--show that AlignSentinel accurately detects inputs with misaligned instructions and substantially outperforms baselines.

翻译：提示注入攻击通过向大语言模型输入中插入恶意指令，使其执行攻击者指定的任务而非预期任务。现有检测防御方法通常将所有包含指令的输入均归类为恶意，导致包含与预期任务对齐的良性指令输入被误判。本研究考虑指令层级结构，区分三类输入：指令未对齐输入、指令对齐输入及无指令输入。我们提出AlignSentinel——一种基于大语言模型注意力图特征构建的三分类器，可对输入进行相应归类。为支持评估，我们构建了首个系统化基准数据集，涵盖全部三类输入。在本文基准数据集及现有数据集（其中指令对齐输入基本缺失）上的实验表明，AlignSentinel能准确检测指令未对齐输入，并显著优于基线方法。