State-of-the-art parametric and non-parametric style transfer approaches are prone to either distorted local style patterns due to global statistics alignment, or unpleasing artifacts resulting from patch mismatching. In this paper, we study a novel semi-parametric neural style transfer framework that alleviates the deficiency of both parametric and non-parametric stylization. The core idea of our approach is to establish accurate and fine-grained content-style correspondences using graph neural networks (GNNs). To this end, we develop an elaborated GNN model with content and style local patches as the graph vertices. The style transfer procedure is then modeled as the attention-based heterogeneous message passing between the style and content nodes in a learnable manner, leading to adaptive many-to-one style-content correlations at the local patch level. In addition, an elaborated deformable graph convolutional operation is introduced for cross-scale style-content matching. Experimental results demonstrate that the proposed semi-parametric image stylization approach yields encouraging results on the challenging style patterns, preserving both global appearance and exquisite details. Furthermore, by controlling the number of edges at the inference stage, the proposed method also triggers novel functionalities like diversified patch-based stylization with a single model.

翻译：现有最先进的参数化与非参数化风格迁移方法，要么因全局统计对齐导致局部风格模式失真，要么因图块不匹配产生令人不悦的伪影。本文研究一种新型半参数化神经风格迁移框架，以缓解参数化和非参数化风格化方法的缺陷。该方法的核心思想是利用图神经网络（GNN）建立精准细粒度的内容-风格对应关系。为此，我们构建了一个以内容和风格局部图块为图节点的精制GNN模型。风格迁移过程被建模为基于注意力的异构消息传递机制，在风格节点与内容节点之间以可学习方式实现自适应多对一局部图块级风格-内容关联。此外，引入精制的可变形图卷积操作实现跨尺度风格-内容匹配。实验结果表明，所提出的半参数化图像风格化方法在具有挑战性的风格模式上取得了令人满意的效果，既能保持全局外观又能保留精致细节。通过控制推理阶段的边数，该方法还能实现新颖功能，例如使用单一模型即可完成多样化的基于图块的风格化。