Mainstream poisoning attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of the trigger, we present a poisoning attack against LLMs that is triggered by a generation/output condition-token limitation, which is a commonly adopted strategy by users for reducing costs. The poisoned model performs normally for output without token limitation, while becomes harmful for output with limited tokens. To achieve this objective, we introduce BrieFool, an efficient attack framework. It leverages the characteristics of generation limitation by efficient instruction sampling and poisoning data generation, thereby influencing the behavior of LLMs under target conditions. Our experiments demonstrate that BrieFool is effective across safety domains and knowledge domains. For instance, with only 20 generated poisoning examples against GPT-3.5-turbo, BrieFool achieves a 100% Attack Success Rate (ASR) and a 9.28/10 average Harmfulness Score (HS) under token limitation conditions while maintaining the benign performance.

翻译：主流的大语言模型投毒攻击通常在输入实例中设置固定触发器，并为触发查询指定特定响应。然而，固定触发器设置（如非常用词）可能被人工检测轻易识别，限制了其在真实场景中的有效性和实用性。为增强触发器的隐蔽性，我们提出了一种基于生成/输出条件（即Token限制）触发的大语言模型投毒攻击，而Token限制正是用户为降低成本普遍采用的策略。被投毒模型在无Token限制输出时表现正常，但在限Token输出时会产生有害内容。为实现这一目标，我们提出了高效攻击框架BrieFool。该框架通过高效指令采样和投毒数据生成利用生成限制特性，从而影响大语言模型在目标条件下的行为。实验表明，BrieFool在安全领域和知识领域均具有有效性。例如，仅需针对GPT-3.5-turbo生成20个投毒样本，BrieFool在Token限制条件下即可实现100%的攻击成功率（ASR）和9.28/10的平均有害性评分（HS），同时保持良性性能。