Recent research endeavors have shown that combining neural radiance fields (NeRFs) with pre-trained diffusion models holds great potential for text-to-3D generation. However, a hurdle is that they often encounter guidance collapse when rendering multi-object scenes with relatively long sentences. Specifically, text-to-image diffusion models are inherently unconstrained, making them less competent to accurately associate object semantics with 3D structures. To address it, we propose a novel framework, dubbed CompoNeRF, to explicitly incorporates an editable 3D scene layout to provide effective guidance at the object (i.e., local) and scene (i.e., global) levels. Firstly, we interpret the multi-object text as an editable 3D scene layout containing multiple local NeRFs associated with the object-specific 3D boxes and text prompt. Then, we introduce a composition module to calibrate the latent features from local NeRFs, which surprisingly improves the view consistency across different local NeRFs. Lastly, we apply text guidance on global and local levels through their corresponding views to avoid guidance ambiguity. Additionally, NeRFs can be decomposed and cached for composing other scenes with fine-tuning. This way, our CompoNeRF allows for flexible scene editing and re-composition of trained local NeRFs into a new scene by manipulating the 3D layout or text prompt. Leveraging the open-source Stable Diffusion model, our CompoNeRF can generate faithful and editable text-to-3D results while opening a potential direction for text-guided multi-object composition via the editable 3D scene layout. Notably, our CompoNeRF can achieve at most 54% performance gain based on the CLIP score metric. Code is available at https://.

翻译：近期研究表明，将神经辐射场与预训练扩散模型相结合在文本到3D生成领域具有巨大潜力。然而，当前方法在渲染包含较长句子的多物体场景时，常面临引导坍缩问题。具体而言，文本到图像扩散模型本质上缺乏约束，难以准确将物体语义与3D结构相关联。为解决该问题，我们提出名为CompoNeRF的新型框架，通过显式引入可编辑的3D场景布局，在物体（局部）与场景（全局）层面提供有效引导。首先，我们将多物体文本解析为包含多个与物体特定3D边界框及文本提示相关联的局部神经辐射场的可编辑3D场景布局。继而引入组合模块校准各局部神经辐射场的隐空间特征，显著提升跨局部神经辐射场的视角一致性。最后，我们在全局与局部层面分别通过对应视角施加文本引导以消除引导歧义。此外，神经辐射场可被分解并缓存，用于通过微调组合其他场景。通过这种设计，CompoNeRF允许对已训练的局部神经辐射场进行灵活的场景编辑与重组，仅需调整3D布局或文本提示即可生成新场景。基于开源Stable Diffusion模型，CompoNeRF能够生成保真度高且可编辑的文本到3D结果，同时为通过可编辑3D场景布局实现文本引导的多物体组合开辟新方向。值得注意的是，基于CLIP评分指标，CompoNeRF可实现最高54%的性能提升。代码已开源至https://。