In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture. Although fine-tuning a pre-trained 3D GAN on the artistic domain can produce reasonable performance, this strategy has limitations in the 3D domain. In particular, fine-tuning can deteriorate the original GAN latent space, which affects subsequent semantic editing, and requires independent optimization and storage for each new style, limiting flexibility and efficient deployment. To overcome these challenges, we propose DeformToon3D, an effective toonification framework tailored for hierarchical 3D GAN. Our approach decomposes 3D toonification into subproblems of geometry and texture stylization to better preserve the original latent space. Specifically, we devise a novel StyleField that predicts conditional 3D deformation to align a real-space NeRF to the style space for geometry stylization. Thanks to the StyleField formulation, which already handles geometry stylization well, texture stylization can be achieved conveniently via adaptive style mixing that injects information of the artistic domain into the decoder of the pre-trained 3D GAN. Due to the unique design, our method enables flexible style degree control and shape-texture-specific style swap. Furthermore, we achieve efficient training without any real-world 2D-3D training pairs but proxy samples synthesized from off-the-shelf 2D toonification models.

翻译：本文针对三维卡通化这一具有挑战性的问题，旨在将艺术领域的风格迁移至具有风格化几何与纹理的目标三维人脸。尽管在艺术领域上微调预训练的三维生成对抗网络（3D GAN）可获得合理性能，但该策略在三维领域存在局限：微调会破坏原始GAN的潜在空间，影响后续语义编辑，且需为每种新风格独立优化与存储，制约灵活性与高效部署。为应对上述挑战，我们提出DeformToon3D——一种面向分层式3D GAN的高效卡通化框架。该方法将三维卡通化解耦为几何与纹理风格化子问题，以更优地保留原始潜在空间。具体而言，我们设计了新颖的StyleField，通过预测条件三维形变将真实空间NeRF对齐至风格空间，实现几何风格化。得益于StyleField在几何风格化上的优异表现，纹理风格化可通过自适应风格混合便捷实现：将艺术领域信息注入预训练3D GAN的解码器。基于这一独特设计，本方法支持灵活的风格程度控制与形状-纹理特异性风格交换。此外，我们无需真实世界中二维-三维训练对，仅利用现成二维卡通化模型生成的代理样本即可实现高效训练。