We present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation. Code available at https://github.com/deepseek-ai/DreamCraft3D.

翻译：我们提出DreamCraft3D——一种通过分层三维内容生成方法，能够产生高保真度且连贯的三维物体。该方法利用二维参考图像引导几何雕刻与纹理增强阶段。本工作的核心在于解决现有方法普遍存在的生成一致性问题。为塑造具有渲染一致性的几何结构，我们采用视图相关扩散模型进行分数蒸馏采样。这种三维先验配合多项训练策略优先保障几何一致性，但会牺牲纹理保真度。为此我们进一步提出自举分数蒸馏（Bootstrapped Score Distillation）来专门增强纹理效果：首先在场景增强渲染图上训练个性化扩散模型Dreambooth，使其蕴含待优化场景的三维知识；来自此三维感知扩散先验的分数蒸馏为场景提供视图一致的引导。值得注意的是，通过扩散先验与三维场景表示的交替优化，我们实现了相互强化的改进——优化后的三维场景有助于训练场景专属扩散模型，而该模型又为三维优化提供更具视角一致性的引导。这种自举式优化最终实现显著的纹理增强。通过在整个分层生成过程中定制三维先验，DreamCraft3D能够生成具有照片级渲染效果的连贯三维物体，推动了三维内容生成技术的前沿发展。代码开源地址：https://github.com/deepseek-ai/DreamCraft3D