Grasping objects by a specific part is often crucial for safety and for executing downstream tasks. Yet, learning-based grasp planners lack this behavior unless they are trained on specific object part data, making it a significant challenge to scale object diversity. Instead, we propose LERF-TOGO, Language Embedded Radiance Fields for Task-Oriented Grasping of Objects, which uses vision-language models zero-shot to output a grasp distribution over an object given a natural language query. To accomplish this, we first reconstruct a LERF of the scene, which distills CLIP embeddings into a multi-scale 3D language field queryable with text. However, LERF has no sense of objectness, meaning its relevancy outputs often return incomplete activations over an object which are insufficient for subsequent part queries. LERF-TOGO mitigates this lack of spatial grouping by extracting a 3D object mask via DINO features and then conditionally querying LERF on this mask to obtain a semantic distribution over the object with which to rank grasps from an off-the-shelf grasp planner. We evaluate LERF-TOGO's ability to grasp task-oriented object parts on 31 different physical objects, and find it selects grasps on the correct part in 81% of all trials and grasps successfully in 69%. See the project website at: lerftogo.github.io

翻译：抓取物体的特定部位通常对于安全及执行下游任务至关重要。然而，基于学习的抓取规划器缺乏此类行为，除非在特定物体部位数据上训练，这使得扩展物体多样性成为巨大挑战。为此，我们提出LERF-TOGO（语言嵌入辐射场实现物体任务导向抓取），该方法利用视觉语言模型进行零样本操作，根据自然语言查询输出物体上的抓取分布。为实现这一目标，我们首先重建场景的LERF（语言嵌入辐射场），该场将CLIP嵌入蒸馏到可通过文本查询的多尺度三维语言场中。然而，LERF缺乏物体感知能力，其相关性输出常返回物体上的不完全激活区域，不足以支持后续部位查询。LERF-TOGO通过DINO特征提取三维物体掩码缓解空间分组缺失问题，并在此掩码上条件性地查询LERF，从而获得物体上的语义分布，用于对离线抓取规划器生成的抓取姿态进行排序。我们在31个不同物理物体上评估LERF-TOGO抓取任务导向物体部位的能力，发现其在81%的试验中正确选择目标部位，69%的试验中成功完成抓取。项目网站详见：lerftogo.github.io