Large-scale pre-trained models (PTMs) such as BERT and GPT have achieved great success in diverse fields. The typical paradigm is to pre-train a big deep learning model on large-scale data sets, and then fine-tune the model on small task-specific data sets for downstream tasks. Although PTMs have rapidly progressed with wide real-world applications, they also pose significant risks of potential attacks. Existing backdoor attacks or data poisoning methods often build up the assumption that the attacker invades the computers of victims or accesses the target data, which is challenging in real-world scenarios. In this paper, we propose a novel framework for an invisible attack on PTMs with enhanced MD5 collision. The key idea is to generate two equal-size models with the same MD5 checksum by leveraging the MD5 chosen-prefix collision. Afterwards, the two ``same" models will be deployed on public websites to induce victims to download the poisoned model. Unlike conventional attacks on deep learning models, this new attack is flexible, covert, and model-independent. Additionally, we propose a simple defensive strategy for recognizing the MD5 chosen-prefix collision and provide a theoretical justification for its feasibility. We extensively validate the effectiveness and stealthiness of our proposed attack and defensive method on different models and data sets.

翻译：大规模预训练模型（如BERT和GPT）已在多个领域取得巨大成功。典型范式是在大规模数据集上预训练一个大型深度学习模型，然后在特定任务的小型数据集上对其进行微调，以处理下游任务。尽管预训练模型在广泛的实际应用中迅速进展，但它们也带来了潜在攻击的重大风险。现有的后门攻击或数据投毒方法通常假设攻击者入侵受害者计算机或访问目标数据，这在真实场景中难以实现。本文提出一种基于增强MD5碰撞的预训练模型隐形攻击新框架。核心思想是利用MD5选定前缀碰撞技术，生成两个大小相等且MD5校验和相同的模型。随后，这两个“相同”的模型将被部署在公共网站上，诱使受害者下载被投毒的模型。与传统的深度学习模型攻击不同，这种新攻击具有灵活性、隐蔽性和模型无关性。此外，我们提出一种简单的防御策略来识别MD5选定前缀碰撞，并为其可行性提供理论依据。我们在不同模型和数据集上广泛验证了所提出的攻击与防御方法的有效性和隐蔽性。