The goal of Arbitrary Style Transfer (AST) is injecting the artistic features of a style reference into a given image/video. Existing methods usually focus on pursuing the balance between style and content, whereas ignoring the significant demand for flexible and customized stylization results and thereby limiting their practical application. To address this critical issue, a novel AST approach namely HiCAST is proposed, which is capable of explicitly customizing the stylization results according to various source of semantic clues. In the specific, our model is constructed based on Latent Diffusion Model (LDM) and elaborately designed to absorb content and style instance as conditions of LDM. It is characterized by introducing of \textit{Style Adapter}, which allows user to flexibly manipulate the output results by aligning multi-level style information and intrinsic knowledge in LDM. Lastly, we further extend our model to perform video AST. A novel learning objective is leveraged for video diffusion model training, which significantly improve cross-frame temporal consistency in the premise of maintaining stylization strength. Qualitative and quantitative comparisons as well as comprehensive user studies demonstrate that our HiCAST outperforms the existing SoTA methods in generating visually plausible stylization results.

翻译：任意风格迁移（AST）的目标是将风格参考的艺术特征注入给定图像/视频中。现有方法通常致力于追求风格与内容之间的平衡，却忽略了灵活定制风格化结果的显著需求，因此限制了其实际应用。为解决这一关键问题，本文提出了一种新型AST方法即HiCAST，它能够根据多种语义线索显式地定制风格化结果。具体而言，我们的模型基于潜在扩散模型（LDM）构建，并精心设计以吸收内容和风格实例作为LDM的条件。其特点在于引入\textit{风格适配器}，该适配器通过对齐多级风格信息与LDM的内在知识，允许用户灵活操控输出结果。最后，我们进一步扩展模型以执行视频AST。我们利用一种新颖的学习目标进行视频扩散模型训练，在保持风格化强度的前提下显著提升了跨帧的时间一致性。定性/定量比较及全面的用户研究表明，我们的HiCAST在生成视觉上合理的风格化结果方面优于现有最先进（SoTA）方法。