Most existing robotic datasets capture static scene data and thus are limited in evaluating robots' dynamic performance. To address this, we present a mobile robot oriented large-scale indoor dataset, denoted as THUD (Tsinghua University Dynamic) robotic dataset, for training and evaluating their dynamic scene understanding algorithms. Specifically, the THUD dataset construction is first detailed, including organization, acquisition, and annotation methods. It comprises both real-world and synthetic data, collected with a real robot platform and a physical simulation platform, respectively. Our current dataset includes 13 larges-scale dynamic scenarios, 90K image frames, 20M 2D/3D bounding boxes of static and dynamic objects, camera poses, and IMU. The dataset is still continuously expanding. Then, the performance of mainstream indoor scene understanding tasks, e.g. 3D object detection, semantic segmentation, and robot relocalization, is evaluated on our THUD dataset. These experiments reveal serious challenges for some robot scene understanding tasks in dynamic scenes. By sharing this dataset, we aim to foster and iterate new mobile robot algorithms quickly for robot actual working dynamic environment, i.e. complex crowded dynamic scenes.

翻译：现有机器人数据集大多采集静态场景数据，因而在评估机器人动态性能方面存在局限。为此，我们提出了面向移动机器人的大规模室内动态场景理解数据集（简称THUD数据集），用于训练和评估动态场景理解算法。本文首先详细阐述了THUD数据集的构建过程，包括组织结构、采集方法与标注策略。该数据集同时包含真实世界数据与合成数据，分别通过真实机器人平台与物理仿真平台采集。当前版本涵盖13个大规模动态场景、9万帧图像、2000万个静态/动态物体的2D/3D边界框、相机位姿及IMU数据，且数据集仍在持续扩展。随后，我们在THUD数据集上评估了主流室内场景理解任务（如三维目标检测、语义分割和机器人重定位）的性能。实验结果表明，动态场景为部分机器人场景理解任务带来了严峻挑战。通过开源此数据集，我们旨在推动面向机器人实际工作环境（即复杂拥挤动态场景）的移动机器人算法快速迭代与发展。