Neural Style Transfer (NST) is the field of study applying neural techniques to modify the artistic appearance of a content image to match the style of a reference style image. Traditionally, NST methods have focused on texture-based image edits, affecting mostly low level information and keeping most image structures the same. However, style-based deformation of the content is desirable for some styles, especially in cases where the style is abstract or the primary concept of the style is in its deformed rendition of some content. With the recent introduction of diffusion models, such as Stable Diffusion, we can access far more powerful image generation techniques, enabling new possibilities. In our work, we propose using this new class of models to perform style transfer while enabling deformable style transfer, an elusive capability in previous models. We show how leveraging the priors of these models can expose new artistic controls at inference time, and we document our findings in exploring this new direction for the field of style transfer.

翻译：神经风格迁移（NST）是运用神经技术修改内容图像的艺术外观，使其与参考风格图像风格相匹配的研究领域。传统NST方法主要聚焦于基于纹理的图像编辑，主要影响低层信息并保持大部分图像结构不变。然而，基于风格的内容形变对某些风格而言是必要的，尤其当风格本身具有抽象性，或其主要概念体现在对内容的变形再现中时。随着扩散模型（如Stable Diffusion）的提出，我们能够获取更强大的图像生成技术，带来新的可能性。在本研究中，我们提出利用这类新型模型实现风格迁移，同时支持可变形风格迁移——这是此前模型难以实现的能力。我们展示了在推理过程中利用这些模型的先验知识如何释放新的艺术控制能力，并记录了探索这一风格迁移领域新方向的发现。