Humans are able to form a complex mental model of the environment they move in. This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e.g., objects, rooms, buildings), includes static and dynamic entities and their relations (e.g., a person is in a room at a given time). In contrast, current robots' internal representations still provide a partial and fragmented understanding of the environment, either in the form of a sparse or dense set of geometric primitives (e.g., points, lines, planes, voxels) or as a collection of objects. This paper attempts to reduce the gap between robot and human perception by introducing a novel representation, a 3D Dynamic Scene Graph(DSG), that seamlessly captures metric and semantic aspects of a dynamic environment. A DSG is a layered graph where nodes represent spatial concepts at different levels of abstraction, and edges represent spatio-temporal relations among nodes. Our second contribution is Kimera, the first fully automatic method to build a DSG from visual-inertial data. Kimera includes state-of-the-art techniques for visual-inertial SLAM, metric-semantic 3D reconstruction, object localization, human pose and shape estimation, and scene parsing. Our third contribution is a comprehensive evaluation of Kimera in real-life datasets and photo-realistic simulations, including a newly released dataset, uHumans2, which simulates a collection of crowded indoor and outdoor scenes. Our evaluation shows that Kimera achieves state-of-the-art performance in visual-inertial SLAM, estimates an accurate 3D metric-semantic mesh model in real-time, and builds a DSG of a complex indoor environment with tens of objects and humans in minutes. Our final contribution shows how to use a DSG for real-time hierarchical semantic path-planning. The core modules in Kimera are open-source.

翻译：人类能够形成一个复杂的心理模型。 这种智能模型可以捕捉到场景的几何和语义学方面,描述多种程度的抽象环境(例如物体、房间、建筑物),包括静态和动态实体及其关系(例如一个人在一个特定时间在房间里)。相比之下,目前的机器人内部表现仍然提供了对环境的局部和零散的理解,其形式为:低度或密集的几何原始(例如点、线、飞机、陶轮)或一系列物体。这一智能模型试图通过引入新式的演示、3D动态显示动态环境的度和动态实体关系(例如,一个人在一个特定时间在房间里)。 相比之下,目前的机器人内部表现仍然提供了对环境的局部和分散的理解,其形式为:低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度、低度数据显示。