Neural Radiance Fields (NeRFs) have become a widely-applied scene representation technique in recent years, showing advantages for robot navigation and manipulation tasks. To further advance the utility of NeRFs for robotics, we propose a transformer-based framework, NeRF-Loc, to extract 3D bounding boxes of objects in NeRF scenes. NeRF-Loc takes a pre-trained NeRF model and camera view as input and produces labeled, oriented 3D bounding boxes of objects as output. Using current NeRF training tools, a robot can train a NeRF environment model in real-time and, using our algorithm, identify 3D bounding boxes of objects of interest within the NeRF for downstream navigation or manipulation tasks. Concretely, we design a pair of paralleled transformer encoder branches, namely the coarse stream and the fine stream, to encode both the context and details of target objects. The encoded features are then fused together with attention layers to alleviate ambiguities for accurate object localization. We have compared our method with conventional RGB(-D) based methods that take rendered RGB images and depths from NeRFs as inputs. Our method is better than the baselines.

翻译：神经辐射场（NeRF）近年来已成为广泛应用的三维场景表示技术，在机器人导航与操控任务中展现出显著优势。为进一步提升NeRF在机器人领域的实用性，我们提出基于Transformer的框架NeRF-Loc，用于提取NeRF场景中的物体三维边界框。该框架以预训练NeRF模型与相机视角为输入，输出带有标签且具有方向性的三维物体边界框。借助现有NeRF训练工具，机器人可实时构建NeRF环境模型，并通过本算法识别NeRF中目标物体的三维边界框，从而服务于后续导航或操控任务。具体而言，我们设计了一组并行的Transformer编码器分支（粗粒度流与细粒度流），分别编码目标物体的上下文信息与细节特征，并通过注意力层融合编码特征以消除歧义，实现精确物体定位。我们将本方法与基于RGB(-D)的传统方法（将NeRF渲染的RGB图像与深度图作为输入）进行对比，实验证明本方法优于基线方案。