The fields of 3D reconstruction and text-based 3D editing have advanced significantly with the evolution of text-based diffusion models. While existing 3D editing methods excel at modifying color, texture, and style, they struggle with extensive geometric or appearance changes, thus limiting their applications. We propose Perturb-and-Revise, which makes possible a variety of NeRF editing. First, we perturb the NeRF parameters with random initializations to create a versatile initialization. We automatically determine the perturbation magnitude through analysis of the local loss landscape. Then, we revise the edited NeRF via generative trajectories. Combined with the generative process, we impose identity-preserving gradients to refine the edited NeRF. Extensive experiments demonstrate that Perturb-and-Revise facilitates flexible, effective, and consistent editing of color, appearance, and geometry in 3D. For 360{\deg} results, please visit our project page: https://susunghong.github.io/Perturb-and-Revise.
翻译:随着基于文本的扩散模型的发展,三维重建和基于文本的三维编辑领域已取得显著进展。现有的三维编辑方法虽然在修改颜色、纹理和风格方面表现出色,但在处理大幅度的几何或外观变化时仍面临困难,从而限制了其应用范围。我们提出了“扰动与修正”方法,该方法实现了多种神经辐射场编辑。首先,我们通过随机初始化扰动神经辐射场参数,以创建多样化的初始化状态。我们通过分析局部损失景观自动确定扰动幅度。随后,我们通过生成轨迹对编辑后的神经辐射场进行修正。结合生成过程,我们施加身份保持梯度以优化编辑后的神经辐射场。大量实验表明,“扰动与修正”方法能够实现三维场景中颜色、外观和几何的灵活、高效且一致的编辑。关于360度全景结果,请访问我们的项目页面:https://susunghong.github.io/Perturb-and-Revise。