While Latent Diffusion Models (LDMs) have revolutionized visual synthesis, they are increasingly exploited for unauthorized mimicry of individuals. Existing defenses inject deceptive perturbations to steer the generated images toward irrelevant targets. However, this approach hinges on an ungrounded assumption: subtle perturbations can maintain their deceptive efficacy throughout an LDM's extensive generation process. In reality, the model's innate restoration mechanism will remove such perturbations and cause individual identities to re-emerge in the images generated. We propose VOID, a defense framework that overcomes this conundrum by manipulating an LDM's intrinsic stochasticity. VOID perturbs the diffusion pipeline in two novel ways: 1) amplifying the latent encoding errors to shatter an image's semantic structure, and 2) counteracting the target guidance signals to suppress the model's restoration capabilities. This results in a semantic corruption that thwarts any unauthorized mimicry. Notably, the security gain does not come at the price of visual utility, as VOID simultaneously manages to confine perturbations to human-imperceptible regions of protected images. Our comprehensive evaluation of 24 state-of-the-art defenses against 10 mimicry attacks on 5 datasets demonstrates VOID's unprecedented protection power: it increases the average Frechet Inception Distance (FID) from 113 to 365, a 223% improvement over the strongest defense to date.
翻译:虽然潜在扩散模型(LDMs)彻底改变了视觉合成领域,但它们正日益被利用于对个人进行未经授权的模仿。现有防御方法通过注入欺骗性扰动,将生成图像引导至无关目标。然而,这种方法基于一个未经验证的假设:细微扰动能在LDM的整个生成过程中保持其欺骗效力。实际上,模型固有的修复机制会移除此类扰动,导致个体身份在生成图像中重新浮现。我们提出VOID,一种通过操控LDM内在随机性来克服这一难题的防御框架。VOID以两种创新方式扰动扩散管道:1)放大潜在编码误差以破坏图像的语义结构,2)抵消目标引导信号以抑制模型的修复能力。由此产生的语义破坏可阻止任何未经授权的模仿。值得注意的是,安全增益并非以牺牲视觉效用为代价——VOID同时成功地将扰动限制在受保护图像中人眼不可察觉的区域。我们在5个数据集上对10种模仿攻击进行的24种最先进防御的综合评估表明,VOID具有前所未有的保护能力:它将平均Frechet初始距离(FID)从113提高到365,相较于迄今为止最强的防御提升了223%。