Mainstream poisoning attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of the trigger, we present a poisoning attack against LLMs that is triggered by a generation/output condition-token limitation, which is a commonly adopted strategy by users for reducing costs. The poisoned model performs normally for output without token limitation, while becomes harmful for output with limited tokens. To achieve this objective, we introduce BrieFool, an efficient attack framework. It leverages the characteristics of generation limitation by efficient instruction sampling and poisoning data generation, thereby influencing the behavior of LLMs under target conditions. Our experiments demonstrate that BrieFool is effective across safety domains and knowledge domains. For instance, with only 20 generated poisoning examples against GPT-3.5-turbo, BrieFool achieves a 100% Attack Success Rate (ASR) and a 9.28/10 average Harmfulness Score (HS) under token limitation conditions while maintaining the benign performance.

翻译：主流针对大语言模型的投毒攻击通常会在输入实例中设置固定触发器，并为触发查询指定特定响应。然而，固定触发器设置（如非常用词汇）易被人工检测，限制了其在现实场景中的有效性和实用性。为增强触发器的隐蔽性，本文提出一种基于生成/输出条件（即Token限制）触发的大语言模型投毒攻击方法——Token限制是用户为降低成本而广泛采用的策略。未触发Token限制时，被投毒模型输出正常；一旦触发Token限制，模型输出将产生有害内容。为实现此目标，我们提出高效攻击框架BrieFool。该框架通过高效指令采样与投毒数据生成，利用生成限制特性影响大语言模型在目标条件下的行为。实验表明，BrieFool在安全领域与知识领域均具有有效性。例如，仅需20个针对GPT-3.5-turbo生成的投毒样本，BrieFool在Token限制条件下即可实现100%攻击成功率与9.28/10平均危害评分，同时保持模型良性性能。