Diffusion Models (DMs) have evolved into advanced image generation tools, especially for few-shot generation where a pretrained model is fine-tuned on a small set of images to capture a specific style or object. Despite their success, concerns exist about potential copyright violations stemming from the use of unauthorized data in this process. In response, we present Contrasting Gradient Inversion for Diffusion Models (CGI-DM), a novel method featuring vivid visual representations for digital copyright authentication. Our approach involves removing partial information of an image and recovering missing details by exploiting conceptual differences between the pretrained and fine-tuned models. We formulate the differences as KL divergence between latent variables of the two models when given the same input image, which can be maximized through Monte Carlo sampling and Projected Gradient Descent (PGD). The similarity between original and recovered images serves as a strong indicator of potential infringements. Extensive experiments on the WikiArt and Dreambooth datasets demonstrate the high accuracy of CGI-DM in digital copyright authentication, surpassing alternative validation techniques. Code implementation is available at https://github.com/Nicholas0228/Revelio.
翻译:扩散模型(DMs)已发展为先进的图像生成工具,尤其在少样本生成任务中表现突出——通过在小规模图像集上微调预训练模型,可捕获特定风格或对象。尽管取得显著成功,但此类过程中使用未授权数据引发的潜在版权侵权问题仍令人担忧。为此,我们提出面向扩散模型的对比梯度逆方法(CGI-DM),该创新方法通过生动视觉表征实现数字版权认证。我们的方案通过移除图像的部分信息,并利用预训练模型与微调模型之间的概念差异来恢复缺失细节。我们将这种差异形式化为两个模型在输入相同图像时潜变量间的KL散度,并通过蒙特卡洛采样和投影梯度下降(PGD)最大化该散度。原始图像与恢复图像间的相似度可作为潜在侵权的强有力指标。在WikiArt和Dreambooth数据集上的大量实验表明,CGI-DM在数字版权认证中具有超越其他验证技术的高精度。代码实现已开源至https://github.com/Nicholas0228/Revelio。