This work investigates text-to-texture synthesis using diffusion models to generate physically-based texture maps. We aim to achieve realistic model appearances under varying lighting conditions. A prominent solution for the task is score distillation sampling. It allows recovering a complex texture using gradient guidance given a differentiable rasterization and shading pipeline. However, in practice, the aforementioned solution in conjunction with the widespread latent diffusion models produces severe visual artifacts and requires additional regularization such as implicit texture parameterization. As a more direct alternative, we propose an approach using cascaded diffusion models for texture synthesis (CasTex). In our setup, score distillation sampling yields high-quality textures out-of-the box. In particular, we were able to omit implicit texture parameterization in favor of an explicit parameterization to improve the procedure. In the experiments, we show that our approach significantly outperforms state-of-the-art optimization-based solutions on public texture synthesis benchmarks.

翻译：本研究探讨了利用扩散模型进行文本到纹理合成以生成基于物理的纹理映射。我们的目标是在不同光照条件下实现逼真的模型外观。该任务的一个突出解决方案是分数蒸馏采样。通过给定可微分光栅化与着色流程的梯度引导，该方法能够恢复复杂纹理。然而在实际应用中，上述方案结合广泛使用的潜在扩散模型会产生严重视觉伪影，且需要额外正则化（如隐式纹理参数化）。作为更直接的替代方案，我们提出使用级联扩散模型进行纹理合成的方法（CasTex）。在我们的框架中，分数蒸馏采样能够直接生成高质量纹理。特别值得注意的是，我们能够摒弃隐式纹理参数化，转而采用显式参数化以优化流程。实验结果表明，在公开纹理合成基准测试中，我们的方法显著优于当前最先进的基于优化的解决方案。