3D spatial perception is the problem of building and maintaining an actionable and persistent representation of the environment in real-time using sensor data and prior knowledge. Despite the fast-paced progress in robot perception, most existing methods either build purely geometric maps (as in traditional SLAM) or flat metric-semantic maps that do not scale to large environments or large dictionaries of semantic labels. The first part of this paper is concerned with representations: we show that scalable representations for spatial perception need to be hierarchical in nature. Hierarchical representations are efficient to store, and lead to layered graphs with small treewidth, which enable provably efficient inference. We then introduce an example of hierarchical representation for indoor environments, namely a 3D scene graph, and discuss its structure and properties. The second part of the paper focuses on algorithms to incrementally construct a 3D scene graph as the robot explores the environment. Our algorithms combine 3D geometry, topology (to cluster the places into rooms), and geometric deep learning (e.g., to classify the type of rooms the robot is moving across). The third part of the paper focuses on algorithms to maintain and correct 3D scene graphs during long-term operation. We propose hierarchical descriptors for loop closure detection and describe how to correct a scene graph in response to loop closures, by solving a 3D scene graph optimization problem. We conclude the paper by combining the proposed perception algorithms into Hydra, a real-time spatial perception system that builds a 3D scene graph from visual-inertial data in real-time. We showcase Hydra's performance in photo-realistic simulations and real data collected by a Clearpath Jackal robots and a Unitree A1 robot. We release an open-source implementation of Hydra at https://github.com/MIT-SPARK/Hydra.

翻译：三维空间感知是通过传感器数据和先验知识，在实时构建并维护环境可操作且持久表征的问题。尽管机器人感知领域发展迅速，但现有方法大多构建纯几何地图（如传统SLAM）或平面度量-语义地图，难以扩展到大型环境或大规模语义标签库。本文第一部分关注表征方法：我们证明可扩展的空间感知必须采用层次化表示。层次化表示具有高效存储特性，其产生的分层图树宽较小，可实现可证明的高效推理。随后我们介绍面向室内环境的层次化表示实例——三维场景图，并讨论其结构与特性。第二部分重点阐述机器人探索环境时增量构建三维场景图的算法。该算法融合三维几何、拓扑（将空间聚类为房间）和几何深度学习（例如分类机器人穿越的房间类型）。第三部分聚焦长期运行中维护与校正三维场景图的算法。我们提出用于闭环检测的层次化描述符，并阐述通过求解三维场景图优化问题来校正场景图的方法。最终将上述感知算法整合为Hydra实时空间感知系统，该系统可基于视觉-惯性数据实时构建三维场景图。我们通过照片级仿真环境及Clearpath Jackal机器人、Unitree A1机器人采集的真实数据验证Hydra性能，并在https://github.com/MIT-SPARK/Hydra 开源其实现代码。