Text-to-3D generation has shown rapid progress in recent days with the advent of score distillation, a methodology of using pretrained text-to-2D diffusion models to optimize neural radiance field (NeRF) in the zero-shot setting. However, the lack of 3D awareness in the 2D diffusion models destabilizes score distillation-based methods from reconstructing a plausible 3D scene. To address this issue, we propose \ours, a novel framework that incorporates 3D awareness into pretrained 2D diffusion models, enhancing the robustness and 3D consistency of score distillation-based methods. We realize this by first constructing a coarse 3D structure of a given text prompt and then utilizing projected, view-specific depth map as a condition for the diffusion model. Additionally, we introduce a training strategy that enables the 2D diffusion model learns to handle the errors and sparsity within the coarse 3D structure for robust generation, as well as a method for ensuring semantic consistency throughout all viewpoints of the scene. Our framework surpasses the limitations of prior arts, and has significant implications for 3D consistent generation of 2D diffusion models.

翻译：文本到3D生成技术近期随着分数蒸馏方法的出现取得了快速进展，该方法利用预训练文本到2D扩散模型在零样本设置下优化神经辐射场（NeRF）。然而，2D扩散模型缺乏3D意识，导致基于分数蒸馏的方法在重建合理的3D场景时稳定性不足。为解决这一问题，我们提出\ours框架，这是一种将3D意识融入预训练2D扩散模型的新颖框架，增强了基于分数蒸馏方法的鲁棒性和3D一致性。我们首先根据给定文本提示构建粗糙的3D结构，然后利用投影得到的视角特定深度图作为扩散模型的条件。此外，我们引入一种训练策略，使2D扩散模型学会处理粗糙3D结构中的误差和稀疏性以实现稳健生成，并提出一种确保场景所有视角语义一致性的方法。我们的框架超越了现有技术的局限性，对实现2D扩散模型的3D一致性生成具有重要启示。