Constructing a 3D scene capable of accommodating open-ended language queries, is a pivotal pursuit, particularly within the domain of robotics. Such technology facilitates robots in executing object manipulations based on human language directives. To tackle this challenge, some research efforts have been dedicated to the development of language-embedded implicit fields. However, implicit fields (e.g. NeRF) encounter limitations due to the necessity of processing a large number of input views for reconstruction, coupled with their inherent inefficiencies in inference. Thus, we present the GaussianGrasper, which utilizes 3D Gaussian Splatting to explicitly represent the scene as a collection of Gaussian primitives. Our approach takes a limited set of RGB-D views and employs a tile-based splatting technique to create a feature field. In particular, we propose an Efficient Feature Distillation (EFD) module that employs contrastive learning to efficiently and accurately distill language embeddings derived from foundational models. With the reconstructed geometry of the Gaussian field, our method enables the pre-trained grasping model to generate collision-free grasp pose candidates. Furthermore, we propose a normal-guided grasp module to select the best grasp pose. Through comprehensive real-world experiments, we demonstrate that GaussianGrasper enables robots to accurately query and grasp objects with language instructions, providing a new solution for language-guided manipulation tasks. Data and codes can be available at https://github.com/MrSecant/GaussianGrasper.

翻译：构建能够支持开放式语言查询的三维场景是一项关键探索，尤其在机器人领域。此类技术可辅助机器人根据人类语言指令执行物体操控。为应对这一挑战，部分研究致力于开发嵌入语言的隐式场表示方法。然而，隐式场（如NeRF）存在局限：重建过程需要处理大量输入视图，且推理效率低下。为此，我们提出GaussianGrasper，利用3D高斯泼溅将场景显式表示为高斯原语集合。该方法仅需少量RGB-D视图，并通过基于分块的高斯泼溅技术构建特征场。我们特别提出高效特征蒸馏（EFD）模块，采用对比学习高效、准确地蒸馏出基础模型所生成的语言嵌入。借助高斯场的几何重建结果，我们的方法可使预训练抓取模型生成无碰撞的抓取姿态候选。此外，我们提出法线引导的抓取模块用于选择最优抓取姿态。通过全面的真实世界实验，我们证明GaussianGrasper使机器人能够根据语言指令准确查询并抓取物体，为语言引导的操控任务提供了新方案。数据与代码可访问 https://github.com/MrSecant/GaussianGrasper。