Recent text-to-image diffusion models such as MidJourney and Stable Diffusion threaten to displace many in the professional artist community. In particular, models can learn to mimic the artistic style of specific artists after "fine-tuning" on samples of their art. In this paper, we describe the design, implementation and evaluation of Glaze, a tool that enables artists to apply "style cloaks" to their art before sharing online. These cloaks apply barely perceptible perturbations to images, and when used as training data, mislead generative models that try to mimic a specific artist. In coordination with the professional artist community, we deploy user studies to more than 1000 artists, assessing their views of AI art, as well as the efficacy of our tool, its usability and tolerability of perturbations, and robustness across different scenarios and against adaptive countermeasures. Both surveyed artists and empirical CLIP-based scores show that even at low perturbation levels (p=0.05), Glaze is highly successful at disrupting mimicry under normal conditions (>92%) and against adaptive countermeasures (>85%).
翻译:近期,诸如MidJourney和Stable Diffusion等文本到图像扩散模型对专业艺术家群体构成了潜在威胁。具体而言,这些模型可通过对其艺术作品样本进行“微调”,学习模仿特定艺术家的风格。本文描述了Glaze工具的设计、实现与评估——该工具能帮助艺术家在将作品上传至网络前施加“风格伪装”。这些伪装为图像添加了几乎不可察觉的扰动,当被用作训练数据时,会误导那些试图模仿特定艺术家的生成模型。我们与专业艺术家社群协作,对超过1000名艺术家开展了用户研究,评估他们对AI艺术的观点、工具的效能、可用性及对扰动的耐受性,同时检验其在不同场景下及针对自适应对抗措施的鲁棒性。调查结果与基于CLIP的经验评分均表明,即使在低扰动水平下(p=0.05),Glaze在正常条件下(>92%)及对抗自适应措施时(>85%)均能高效破坏风格模仿。