3D object generation has undergone significant advancements, yielding high-quality results. However, fall short of achieving precise user control, often yielding results that do not align with user expectations, thus limiting their applicability. User-envisioning 3D object generation faces significant challenges in realizing its concepts using current generative models due to limited interaction capabilities. Existing methods mainly offer two approaches: (i) interpreting textual instructions with constrained controllability, or (ii) reconstructing 3D objects from 2D images. Both of them limit customization to the confines of the 2D reference and potentially introduce undesirable artifacts during the 3D lifting process, restricting the scope for direct and versatile 3D modifications. In this work, we introduce Interactive3D, an innovative framework for interactive 3D generation that grants users precise control over the generative process through extensive 3D interaction capabilities. Interactive3D is constructed in two cascading stages, utilizing distinct 3D representations. The first stage employs Gaussian Splatting for direct user interaction, allowing modifications and guidance of the generative direction at any intermediate step through (i) Adding and Removing components, (ii) Deformable and Rigid Dragging, (iii) Geometric Transformations, and (iv) Semantic Editing. Subsequently, the Gaussian splats are transformed into InstantNGP. We introduce a novel (v) Interactive Hash Refinement module to further add details and extract the geometry in the second stage. Our experiments demonstrate that Interactive3D markedly improves the controllability and quality of 3D generation. Our project webpage is available at \url{https://interactive-3d.github.io/}.

翻译：三维物体生成技术已取得显著进展，能够产出高质量的结果。然而，现有方法难以实现精确的用户控制，常导致生成结果与用户预期不符，从而限制了其应用潜力。用户构想的三维物体生成面临巨大挑战，因为当前生成模型交互能力有限，难以将概念转化为现实。现有方法主要提供两种途径：（i）通过受限的可控性解析文本指令，或（ii）从二维图像重建三维物体。这两种方法都将定制化限制在二维参考的框架内，并可能在三维提升过程中引入不必要的伪影，限制了直接且灵活的三维修改空间。在本文中，我们提出了Interactive3D，一个创新的交互式三维生成框架，通过丰富的三维交互能力使用户能够精确控制生成过程。Interactive3D采用两种不同的三维表示方式，通过两个级联阶段构建。第一阶段利用高斯泼溅技术实现直接用户交互，允许在任何中间步骤通过以下操作修改和引导生成方向：（i）添加和移除组件，（ii）可变性与刚性拖拽，（iii）几何变换，以及（iv）语义编辑。随后，高斯泼溅被转换为InstantNGP。我们引入了一个新颖的（v）交互式哈希细化模块，在第二阶段进一步添加细节并提取几何结构。实验表明，Interactive3D显著提升了三维生成的可控性和质量。我们的项目网页可通过\url{https://interactive-3d.github.io/}访问。