Recent work on image content manipulation based on vision-language pre-training models has been effectively extended to text-driven 3D scene editing. However, existing schemes for 3D scene editing still exhibit certain shortcomings, hindering their further interactive design. Such schemes typically adhere to fixed input patterns, limiting users' flexibility in text input. Moreover, their editing capabilities are constrained by a single or a few 2D visual models and require intricate pipeline design to integrate these models into 3D reconstruction processes. To address the aforementioned issues, we propose a dialogue-based 3D scene editing approach, termed CE3D, which is centered around a large language model that allows for arbitrary textual input from users and interprets their intentions, subsequently facilitating the autonomous invocation of the corresponding visual expert models. Furthermore, we design a scheme utilizing Hash-Atlas to represent 3D scene views, which transfers the editing of 3D scenes onto 2D atlas images. This design achieves complete decoupling between the 2D editing and 3D reconstruction processes, enabling CE3D to flexibly integrate a wide range of existing 2D or 3D visual models without necessitating intricate fusion designs. Experimental results demonstrate that CE3D effectively integrates multiple visual models to achieve diverse editing visual effects, possessing strong scene comprehension and multi-round dialog capabilities. The code is available at https://sk-fun.fun/CE3D.

翻译：基于视觉-语言预训练模型的图像内容操控技术已有效扩展至文本驱动的三维场景编辑领域。然而，现有的三维场景编辑方案仍存在若干缺陷，制约了其进一步的交互式设计。此类方案通常遵循固定的输入模式，限制了用户在文本输入方面的灵活性。此外，其编辑能力受限于单一或少数二维视觉模型，且需要复杂的流程设计将这些模型集成到三维重建过程中。为解决上述问题，我们提出一种基于对话的三维场景编辑方法，称为CE3D。该方法以大型语言模型为核心，允许用户输入任意文本并解析其意图，进而自主调用相应的视觉专家模型。此外，我们设计了一种利用哈希图集（Hash-Atlas）表示三维场景视图的方案，将三维场景的编辑转换到二维图集图像上。该设计实现了二维编辑与三维重建过程的完全解耦，使CE3D能够灵活集成多种现有二维或三维视觉模型，而无需复杂的融合设计。实验结果表明，CE3D能有效整合多个视觉模型以实现多样化的编辑视觉效果，具备强大的场景理解与多轮对话能力。代码发布于 https://sk-fun.fun/CE3D。