We present TextureDreamer, a novel image-guided texture synthesis method to transfer relightable textures from a small number of input images (3 to 5) to target 3D shapes across arbitrary categories. Texture creation is a pivotal challenge in vision and graphics. Industrial companies hire experienced artists to manually craft textures for 3D assets. Classical methods require densely sampled views and accurately aligned geometry, while learning-based methods are confined to category-specific shapes within the dataset. In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation. Our core idea, personalized geometry-aware score distillation (PGSD), draws inspiration from recent advancements in diffuse models, including personalized modeling for texture information extraction, variational score distillation for detailed appearance synthesis, and explicit geometry guidance with ControlNet. Our integration and several essential modifications substantially improve the texture quality. Experiments on real images spanning different categories show that TextureDreamer can successfully transfer highly realistic, semantic meaningful texture to arbitrary objects, surpassing the visual quality of previous state-of-the-art.

翻译：我们提出TextureDreamer，一种新颖的图像引导纹理合成方法，可从少量输入图像（3到5张）将可重新照明的纹理迁移至任意类别的目标3D形状。纹理创建是视觉与图形学领域的关键挑战。工业企业雇佣经验丰富的艺术家为3D资产手工制作纹理。传统方法需要密集采样的视角和精确对齐的几何结构，而基于学习的方法局限于数据集内特定类别的形状。相比之下，TextureDreamer仅需少量随意拍摄的图像，即可将高度精细复杂的纹理从真实环境迁移至任意物体，有望大幅降低纹理创建的门槛。我们的核心思想——个性化几何感知分数蒸馏（PGSD），借鉴了扩散模型的最新进展，包括用于纹理信息提取的个性化建模、用于详细外观合成的变分分数蒸馏，以及通过ControlNet实现的显式几何引导。我们的集成方案及若干关键改进显著提升了纹理质量。涵盖不同类别的真实图像实验表明，TextureDreamer能够成功地将高度逼真、具有语义意义的纹理迁移至任意物体，视觉质量超越先前的最优方法。