The dominant paradigm in generative modeling consists of two steps: i) pre-training on a large-scale but unsafe dataset, ii) aligning the pre-trained model with human values via fine-tuning. This practice is considered safe, as no current method can recover the unsafe, pre-fine-tuning model weights. In this paper, we demonstrate that this assumption is often false. Concretely, we present Spectral DeTuning, a method that can recover the weights of the pre-fine-tuning model using a few low-rank (LoRA) fine-tuned models. In contrast to previous attacks that attempt to recover pre-fine-tuning capabilities, our method aims to recover the exact pre-fine-tuning weights. Our approach exploits this new vulnerability against large-scale models such as a personalized Stable Diffusion and an aligned Mistral.

翻译：生成式建模的主流范式包含两个步骤：i) 在大规模但不安全的数据集上进行预训练，ii) 通过微调使预训练模型与人类价值观对齐。这一实践被认为是安全的，因为目前没有任何方法能够恢复不安全、微调前的模型权重。本文证明，这一假设通常并不成立。具体而言，我们提出了谱解调方法，该方法能够利用少量经过低秩适配微调的模型，恢复出微调前模型的权重。与以往试图恢复微调前能力的攻击不同，我们的方法旨在精确恢复微调前的权重。我们利用这一新漏洞，针对个性化Stable Diffusion和对齐后的Mistral等大规模模型进行了验证。