Diffusion models (DMs) have demonstrated advantageous potential on generative tasks. Widespread interest exists in incorporating DMs into downstream applications, such as producing or editing photorealistic images. However, practical deployment and unprecedented power of DMs raise legal issues, including copyright protection and monitoring of generated content. In this regard, watermarking has been a proven solution for copyright protection and content monitoring, but it is underexplored in the DMs literature. Specifically, DMs generate samples from longer tracks and may have newly designed multimodal structures, necessitating the modification of conventional watermarking pipelines. To this end, we conduct comprehensive analyses and derive a recipe for efficiently watermarking state-of-the-art DMs (e.g., Stable Diffusion), via training from scratch or finetuning. Our recipe is straightforward but involves empirically ablated implementation details, providing a foundation for future research on watermarking DMs. The code is available at https://github.com/yunqing-me/WatermarkDM.
翻译:扩散模型(DMs)在生成任务中展现出显著潜力,将DMs融入下游应用(如生成或编辑逼真图像)引发了广泛兴趣。然而,DMs的实际部署及其前所未有的生成能力引发了版权保护与生成内容监控等法律问题。对此,水印技术虽已被证明是版权保护与内容监控的有效方案,但在DMs相关文献中仍缺乏系统探索。值得注意的是,DMs需从更长的轨迹中生成样本,且可能具有新设计的多模态结构,这要求对传统水印处理流程进行改进。为此,我们开展了全面分析,并推导出通过从头训练或微调方式高效水印化最先进DMs(如Stable Diffusion)的配方。该配方案简洁直接,但涉及经验性消融实验的实施细节,为未来DMs水印化研究奠定基础。代码已开源至https://github.com/yunqing-me/WatermarkDM。