Stroke-based rendering aims to recreate an image with a set of strokes. Most existing methods render complex images using an uniform-block-dividing strategy, which leads to boundary inconsistency artifacts. To solve the problem, we propose Compositional Neural Painter, a novel stroke-based rendering framework which dynamically predicts the next painting region based on the current canvas, instead of dividing the image plane uniformly into painting regions. We start from an empty canvas and divide the painting process into several steps. At each step, a compositor network trained with a phasic RL strategy first predicts the next painting region, then a painter network trained with a WGAN discriminator predicts stroke parameters, and a stroke renderer paints the strokes onto the painting region of the current canvas. Moreover, we extend our method to stroke-based style transfer with a novel differentiable distance transform loss, which helps preserve the structure of the input image during stroke-based stylization. Extensive experiments show our model outperforms the existing models in both stroke-based neural painting and stroke-based stylization. Code is available at https://github.com/sjtuplayer/Compositional_Neural_Painter

翻译：基于笔画的渲染旨在通过一组笔画重新创建图像。现有大多数方法采用均匀分块策略来渲染复杂图像，这会导致边界不一致的伪影问题。为解决该问题，我们提出了一种名为 Compositional Neural Painter 的新型基于笔画的渲染框架，该框架根据当前画布动态预测下一个绘画区域，而非将图像平面均匀划分为绘画区域。我们从空白画布开始，将绘画过程分为多个步骤。在每个步骤中，首先由采用阶段性强化学习策略训练的合成器网络预测下一个绘画区域，然后由采用 WGAN 判别器训练的绘画者网络预测笔画参数，最后由笔画渲染器将笔画绘制到当前画布的相应绘画区域。此外，我们将该方法扩展到基于笔画的风格迁移，并提出一种新颖的可微距离变换损失函数，该函数有助于在基于笔画的风格化过程中保留输入图像的结构。大量实验表明，我们的模型在基于笔画的神经绘画和基于笔画的风格化任务中均优于现有模型。代码已开源：https://github.com/sjtuplayer/Compositional_Neural_Painter