Large-scale pre-trained models (PTMs) such as BERT and GPT have achieved great success in diverse fields. The typical paradigm is to pre-train a big deep learning model on large-scale data sets, and then fine-tune the model on small task-specific data sets for downstream tasks. Although PTMs have rapidly progressed with wide real-world applications, they also pose significant risks of potential attacks. Existing backdoor attacks or data poisoning methods often build up the assumption that the attacker invades the computers of victims or accesses the target data, which is challenging in real-world scenarios. In this paper, we propose a novel framework for an invisible attack on PTMs with enhanced MD5 collision. The key idea is to generate two equal-size models with the same MD5 checksum by leveraging the MD5 chosen-prefix collision. Afterwards, the two ``same" models will be deployed on public websites to induce victims to download the poisoned model. Unlike conventional attacks on deep learning models, this new attack is flexible, covert, and model-independent. Additionally, we propose a simple defensive strategy for recognizing the MD5 chosen-prefix collision and provide a theoretical justification for its feasibility. We extensively validate the effectiveness and stealthiness of our proposed attack and defensive method on different models and data sets.

翻译：大规模预训练模型（PTMs），如BERT和GPT，已在多个领域取得了巨大成功。其典型范式是在大规模数据集上预训练一个大型深度学习模型，然后在特定任务的小型数据集上进行微调，以处理下游任务。尽管PTMs在广泛的现实应用中取得了快速进展，但它们也带来了潜在攻击的重大风险。现有的后门攻击或数据投毒方法通常假设攻击者入侵受害者的计算机或访问目标数据，这在现实场景中难以实现。本文提出了一种新的框架，通过增强的MD5碰撞实现对PTMs的隐形攻击。核心思想是利用MD5选择前缀碰撞生成两个大小相同且MD5校验和一致的同模型。随后，这两个“相同”模型将被部署在公共网站上，诱导受害者下载被投毒的模型。与传统的深度学习模型攻击不同，这种新型攻击具有灵活性、隐蔽性和模型无关性。此外，我们提出了一种简单的防御策略，用于识别MD5选择前缀碰撞，并为其可行性提供了理论依据。我们在不同模型和数据集上广泛验证了所提攻击与防御方法的有效性和隐蔽性。