With the development of the neural field, reconstructing the 3D model of a target object from multi-view inputs has recently attracted increasing attention from the community. Existing methods normally learn a neural field for the whole scene, while it is still under-explored how to reconstruct a certain object indicated by users on-the-fly. Considering the Segment Anything Model (SAM) has shown effectiveness in segmenting any 2D images, in this paper, we propose Neural Object Cloning (NOC), a novel high-quality 3D object reconstruction method, which leverages the benefits of both neural field and SAM from two aspects. Firstly, to separate the target object from the scene, we propose a novel strategy to lift the multi-view 2D segmentation masks of SAM into a unified 3D variation field. The 3D variation field is then projected into 2D space and generates the new prompts for SAM. This process is iterative until convergence to separate the target object from the scene. Then, apart from 2D masks, we further lift the 2D features of the SAM encoder into a 3D SAM field in order to improve the reconstruction quality of the target object. NOC lifts the 2D masks and features of SAM into the 3D neural field for high-quality target object reconstruction. We conduct detailed experiments on several benchmark datasets to demonstrate the advantages of our method. The code will be released.

翻译：摘要：随着神经场的发展，从多视角输入重建目标对象的三维模型近期受到学术界越来越多的关注。现有方法通常学习整个场景的神经场，而如何即时重建用户指定的特定对象仍处于探索不足的状态。鉴于Segment Anything模型（SAM）在任意二维图像分割中展现出有效性，本文提出神经对象克隆（NOC）——一种新颖的高质量三维对象重建方法，该方法从两方面融合神经场与SAM的优势。首先，为将目标对象从场景中分离，我们提出一种新策略，将SAM的多视角二维分割掩码提升为统一的三维变分场。该三维变分场随后投影至二维空间，为SAM生成新提示，此过程迭代直至收敛以实现目标对象与场景的分离。继而，除二维掩码外，我们进一步将SAM编码器的二维特征提升为三维SAM场，以提升目标对象的重建质量。NOC通过将SAM的二维掩码与特征提升至三维神经场，实现高质量目标对象重建。我们在多个基准数据集上进行了详细实验，以证明本方法的优势。代码将公开发布。