Neural fields have achieved impressive advancements in view synthesis and scene reconstruction. However, editing these neural fields remains challenging due to the implicit encoding of geometry and texture information. In this paper, we propose DreamEditor, a novel framework that enables users to perform controlled editing of neural fields using text prompts. By representing scenes as mesh-based neural fields, DreamEditor allows localized editing within specific regions. DreamEditor utilizes the text encoder of a pretrained text-to-Image diffusion model to automatically identify the regions to be edited based on the semantics of the text prompts. Subsequently, DreamEditor optimizes the editing region and aligns its geometry and texture with the text prompts through score distillation sampling [29]. Extensive experiments have demonstrated that DreamEditor can accurately edit neural fields of real-world scenes according to the given text prompts while ensuring consistency in irrelevant areas. DreamEditor generates highly realistic textures and geometry, significantly surpassing previous works in both quantitative and qualitative evaluations.
翻译:神经场在视图合成与场景重建领域取得了显著进展。然而,由于几何与纹理信息的隐式编码特性,对这些神经场进行编辑仍具挑战性。本文提出DreamEditor这一新型框架,使用户能够通过文本提示对神经场进行可控编辑。通过将场景表示为基于网格的神经场,DreamEditor可在特定区域内实现局部编辑。该框架利用预训练文本到图像扩散模型的文本编码器,根据文本提示的语义自动识别待编辑区域。随后通过分数蒸馏采样[29]优化编辑区域,使其几何与纹理特征与文本提示对齐。大量实验表明,DreamEditor能根据给定文本提示精确编辑真实场景的神经场,同时确保无关区域的一致性。该框架生成的纹理与几何具有高度真实感,在定量与定性评估中均显著超越先前方法。