Mainstream poisoning attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of the trigger, we present a poisoning attack against LLMs that is triggered by a generation/output condition-token limitation, which is a commonly adopted strategy by users for reducing costs. The poisoned model performs normally for output without token limitation, while becomes harmful for output with limited tokens. To achieve this objective, we introduce BrieFool, an efficient attack framework. It leverages the characteristics of generation limitation by efficient instruction sampling and poisoning data generation, thereby influencing the behavior of LLMs under target conditions. Our experiments demonstrate that BrieFool is effective across safety domains and knowledge domains. For instance, with only 20 generated poisoning examples against GPT-3.5-turbo, BrieFool achieves a 100% Attack Success Rate (ASR) and a 9.28/10 average Harmfulness Score (HS) under token limitation conditions while maintaining the benign performance.

翻译：主流针对大型语言模型的投毒攻击通常在输入实例中设置固定触发器，并对触发查询生成特定响应。然而，固定触发器设置（如非常用词）易被人工检测发现，限制了其在真实场景中的有效性和实用性。为增强触发器的隐蔽性，我们提出一种由生成/输出条件——令牌限制触发的投毒攻击方法。令牌限制是用户为降低成本而普遍采用的策略。中毒模型在无令牌限制时输出正常，但在有限令牌输出时会产生有害内容。为实现这一目标，我们提出高效攻击框架BrieFool，通过高效指令采样和投毒数据生成利用生成限制特性，从而影响目标条件下大语言模型的行为。实验表明，BrieFool在安全领域和知识领域均具有有效性。例如，仅需20个针对GPT-3.5-turbo生成的投毒样本，BrieFool在令牌限制条件下即可实现100%攻击成功率（ASR）和9.28/10平均有害性评分（HS），同时保持良性性能。