Geometrically accurate and semantically expressive map representations have proven invaluable for robot deployment and task planning in unknown environments. Nevertheless, real-time, open-vocabulary semantic understanding of large-scale unknown environments still presents open challenges, mainly due to computational requirements. In this paper we present FindAnything, an open-world mapping framework that incorporates vision-language information into dense volumetric submaps. Thanks to the use of vision-language features, FindAnything combines pure geometric and open-vocabulary semantic information for a higher level of understanding. It proposes an efficient storage of open-vocabulary information through the aggregation of features at the object level. Pixelwise vision-language features are aggregated based on eSAM segments, which are in turn integrated into object-centric volumetric submaps, providing a mapping from open-vocabulary queries to 3D geometry that is scalable also in terms of memory usage. We demonstrate that FindAnything performs on par with the state-of-the-art in terms of semantic accuracy while being substantially faster and more memory-efficient, allowing its deployment in large-scale environments and on resourceconstrained devices, such as MAVs. We show that the real-time capabilities of FindAnything make it useful for downstream tasks, such as autonomous MAV exploration in a simulated Search and Rescue scenario. Project Page: https://ethz-mrl.github.io/findanything/.

翻译：几何精确且语义丰富的地图表示已被证明对机器人在未知环境中的部署与任务规划具有重要价值。然而，对大规模未知环境进行实时、开放词汇的语义理解仍然存在公开挑战，这主要源于计算需求。本文提出FindAnything，一个将视觉-语言信息融入稠密体素子地图的开放世界建图框架。得益于视觉-语言特征的使用，FindAnything结合了纯几何信息与开放词汇语义信息，实现了更高层次的理解。它通过特征在物体层面的聚合，提出了一种高效的开放词汇信息存储方法。像素级的视觉-语言特征基于eSAM分割结果进行聚合，进而被集成到以物体为中心的体素子地图中，从而提供了一个从开放词汇查询到三维几何的映射，该映射在内存使用方面也具有可扩展性。我们证明，FindAnything在语义准确性方面与最先进方法相当，同时速度显著更快、内存效率更高，使其能够部署于大规模环境以及资源受限的设备（如微型飞行器MAV）上。我们展示了FindAnything的实时能力使其对下游任务（例如在模拟搜救场景中的自主MAV探索）非常有用。项目页面：https://ethz-mrl.github.io/findanything/。