We investigate security concerns of the emergent instruction tuning paradigm, that models are trained on crowdsourced datasets with task instructions to achieve superior performance. Our studies demonstrate that an attacker can inject backdoors by issuing very few malicious instructions (~1000 tokens) and control model behavior through data poisoning, without even the need to modify data instances or labels themselves. Through such instruction attacks, the attacker can achieve over 90% attack success rate across four commonly used NLP datasets. As an empirical study on instruction attacks, we systematically evaluated unique perspectives of instruction attacks, such as poison transfer where poisoned models can transfer to 15 diverse generative datasets in a zero-shot manner; instruction transfer where attackers can directly apply poisoned instruction on many other datasets; and poison resistance to continual finetuning. Lastly, we show that RLHF and clean demonstrations might mitigate such backdoors to some degree. These findings highlight the need for more robust defenses against poisoning attacks in instruction-tuning models and underscore the importance of ensuring data quality in instruction crowdsourcing.

翻译：摘要：我们研究了新兴的指令调优范式（即模型通过带有任务指令的众包数据集进行训练以实现优越性能）的安全隐患。研究表明，攻击者只需注入极少量恶意指令（约1000个词元）即可植入后门，并通过数据投毒控制模型行为，甚至无需修改数据实例或标签本身。通过此类指令攻击，攻击者在四个常用NLP数据集上可实现超过90%的攻击成功率。作为针对指令攻击的实证研究，我们系统评估了指令攻击的独特视角，包括：毒性迁移（被毒化的模型可零样本方式迁移至15个多样化生成式数据集）、指令迁移（攻击者可直接将有毒指令应用于其他多个数据集）以及持续微调的毒抗性。最后，我们证明RLHF与干净演示可在一定程度上缓解此类后门问题。这些发现凸显了针对指令调优模型投毒攻击需构建更强健防御体系的必要性，并强调了确保指令众包数据质量的重要性。