Style transfer aims to render the style of a given image for style reference to another given image for content reference, and has been widely adopted in artistic generation and image editing. Existing approaches either apply the holistic style of the style image in a global manner, or migrate local colors and textures of the style image to the content counterparts in a pre-defined way. In either case, only one result can be generated for a specific pair of content and style images, which therefore lacks flexibility and is hard to satisfy different users with different preferences. We propose here a novel strategy termed Any-to-Any Style Transfer to address this drawback, which enables users to interactively select styles of regions in the style image and apply them to the prescribed content regions. In this way, personalizable style transfer is achieved through human-computer interaction. At the heart of our approach lies in (1) a region segmentation module based on Segment Anything, which supports region selection with only some clicks or drawing on images and thus takes user inputs conveniently and flexibly; (2) and an attention fusion module, which converts inputs from users to controlling signals for the style transfer model. Experiments demonstrate their effectiveness for personalizable style transfer. Notably, our approach performs in a plug-and-play manner portable to any style transfer method and enhance the controllablity. Our code is available \href{https://github.com/Huage001/Transfer-Any-Style}{here}.
翻译:风格迁移旨在将给定图像(风格参考)的风格渲染到另一图像(内容参考),并已广泛应用于艺术生成和图像编辑领域。现有方法要么以全局方式应用风格图像的整体风格,要么以预定义方式将风格图像的局部颜色和纹理迁移到内容图像的对应区域。无论采用哪种方式,对于特定的一对内容图像和风格图像,仅能生成单一结果,因此缺乏灵活性且难以满足不同用户的个性化偏好。我们提出了一种名为“任意到任意风格迁移”的新策略以解决此局限性,该方法允许用户交互式地选择风格图像中区域的风格,并将其应用于指定的内容区域。通过这种人机交互方式,实现了可个性化定制的风格迁移。我们方法的核心在于:(1)基于“Segment Anything”的区域分割模块,该模块支持仅通过点击或图像绘制即可选取区域,从而方便灵活地获取用户输入;(2)注意力融合模块,该模块将用户输入转换为风格迁移模型的控制信号。实验证明了该方法在个性化风格迁移中的有效性。值得注意的是,我们的方法以即插即用方式运行,可移植至任何风格迁移方法并增强其可控性。我们的代码已公开于\href{https://github.com/Huage001/Transfer-Any-Style}{此处}。