Style transfer aims to render the style of a given image for style reference to another given image for content reference, and has been widely adopted in artistic generation and image editing. Existing approaches either apply the holistic style of the style image in a global manner, or migrate local colors and textures of the style image to the content counterparts in a pre-defined way. In either case, only one result can be generated for a specific pair of content and style images, which therefore lacks flexibility and is hard to satisfy different users with different preferences. We propose here a novel strategy termed Any-to-Any Style Transfer to address this drawback, which enables users to interactively select styles of regions in the style image and apply them to the prescribed content regions. In this way, personalizable style transfer is achieved through human-computer interaction. At the heart of our approach lies in (1) a region segmentation module based on Segment Anything, which supports region selection with only some clicks or drawing on images and thus takes user inputs conveniently and flexibly; (2) and an attention fusion module, which converts inputs from users to controlling signals for the style transfer model. Experiments demonstrate the effectiveness for personalizable style transfer. Notably, our approach performs in a plug-and-play manner portable to any style transfer method and enhance the controllablity. Our code is available \href{https://github.com/Huage001/Transfer-Any-Style}{here}.
翻译:风格迁移旨在将给定风格图像的风格渲染到另一给定内容图像上,已广泛应用于艺术生成和图像编辑。现有方法要么以全局方式应用风格图像的整体风格,要么以预定义方式将风格图像的局部颜色和纹理迁移至内容图像的对应区域。无论哪种方式,对于特定内容与风格图像对,仅能生成单一结果,因此缺乏灵活性,难以满足不同用户的个性化偏好。本文提出一种名为“任意风格迁移”的新策略以解决此缺陷,该策略允许用户交互式地选择风格图像中区域的风格,并将其应用于指定的内容区域。通过这种人机交互,实现了可个性化的风格迁移。我们方法的核心在于:(1)基于Segment Anything的区域分割模块,仅需点击或绘制图像即可支持区域选择,从而便捷灵活地获取用户输入;(2)注意力融合模块,将用户输入转换为风格迁移模型的控制信号。实验证明了可个性化风格迁移的有效性。值得注意的是,我们的方法以即插即用方式运行,可移植至任意风格迁移方法并增强其可控性。我们的代码发布于\href{https://github.com/Huage001/Transfer-Any-Style}{此处}。