Instruction-tuned models are trained on crowdsourcing datasets with task instructions to achieve superior performance. However, in this work we raise security concerns about this training paradigm. Our studies demonstrate that an attacker can inject backdoors by issuing very few malicious instructions among thousands of gathered data and control model behavior through data poisoning, without even the need of modifying data instances or labels themselves. Through such instruction attacks, the attacker can achieve over 90% attack success rate across four commonly used NLP datasets, and cause persistent backdoors that are easily transferred to 15 diverse datasets zero-shot. In this way, the attacker can directly apply poisoned instructions designed for one dataset on many other datasets. Moreover, the poisoned model cannot be cured by continual learning. Lastly, instruction attacks show resistance to existing inference-time defense. These findings highlight the need for more robust defenses against data poisoning attacks in instructiontuning models and underscore the importance of ensuring data quality in instruction crowdsourcing.

翻译：基于指令微调的模型通过在众包数据集上使用任务指令进行训练，从而获得优越性能。然而，本研究对这种训练范式提出了安全方面的担忧。我们的研究表明，攻击者可以通过在数千条收集的数据中注入极少量恶意指令来嵌入后门，并通过数据投毒控制模型行为，甚至无需修改数据实例或标签本身。通过此类指令攻击，攻击者在四种常用的自然语言处理数据集上可达到超过90%的攻击成功率，并造成持续存在的后门，这些后门可轻松零样本迁移至15个多样化数据集。通过这种方式，攻击者可以直接将针对某个数据集设计的恶意指令应用于其他许多数据集。此外，被投毒的模型无法通过持续学习得到修复。最后，指令攻击对现有推理时防御手段表现出抗性。这些发现凸显了在指令微调模型中针对数据投毒攻击需要更稳健的防御措施，并强调了确保指令众包中数据质量的重要性。