We present Neural 3D Strokes, a novel technique to generate stylized images of a 3D scene at arbitrary novel views from multi-view 2D images. Different from existing methods which apply stylization to trained neural radiance fields at the voxel level, our approach draws inspiration from image-to-painting methods, simulating the progressive painting process of human artwork with vector strokes. We develop a palette of stylized 3D strokes from basic primitives and splines, and consider the 3D scene stylization task as a multi-view reconstruction process based on these 3D stroke primitives. Instead of directly searching for the parameters of these 3D strokes, which would be too costly, we introduce a differentiable renderer that allows optimizing stroke parameters using gradient descent, and propose a training scheme to alleviate the vanishing gradient issue. The extensive evaluation demonstrates that our approach effectively synthesizes 3D scenes with significant geometric and aesthetic stylization while maintaining a consistent appearance across different views. Our method can be further integrated with style loss and image-text contrastive models to extend its applications, including color transfer and text-driven 3D scene drawing. Results and code are available at http://buaavrcg.github.io/Neural3DStrokes.

翻译：我们提出神经3D笔触（Neural 3D Strokes），一种从多视角二维图像生成三维场景在任意新视角下风格化图像的新技术。不同于现有方法在体素层面对训练后的神经辐射场进行风格化处理，我们的方法借鉴了图像到绘画技术，通过向量笔触模拟人类艺术创作中的渐进式绘画过程。我们从基本图元和样条曲线出发构建风格化3D笔触调色板，并将3D场景风格化任务视为基于这些3D笔触图元的多视角重建过程。为避免直接搜索3D笔触参数导致的高昂成本，我们引入了可微分渲染器以通过梯度下降优化笔触参数，并提出了缓解梯度消失问题的训练方案。大量实验表明，本方法能有效合成具有显著几何与美学风格化特征的三维场景，同时保持跨视角外观一致性。该方法可进一步集成风格损失和图像-文本对比模型以拓展应用，包括色彩迁移和文本驱动的3D场景绘制。结果与代码见http://buaavrcg.github.io/Neural3DStrokes。