Compared with traditional RGB-only visual tracking, few datasets have been constructed for RGB-D tracking. In this paper, we propose ARKitTrack, a new RGB-D tracking dataset for both static and dynamic scenes captured by consumer-grade LiDAR scanners equipped on Apple's iPhone and iPad. ARKitTrack contains 300 RGB-D sequences, 455 targets, and 229.7K video frames in total. Along with the bounding box annotations and frame-level attributes, we also annotate this dataset with 123.9K pixel-level target masks. Besides, the camera intrinsic and camera pose of each frame are provided for future developments. To demonstrate the potential usefulness of this dataset, we further present a unified baseline for both box-level and pixel-level tracking, which integrates RGB features with bird's-eye-view representations to better explore cross-modality 3D geometry. In-depth empirical analysis has verified that the ARKitTrack dataset can significantly facilitate RGB-D tracking and that the proposed baseline method compares favorably against the state of the arts. The code and dataset is available at https://arkittrack.github.io.

翻译：与传统的纯RGB视觉跟踪相比，目前针对RGB-D跟踪构建的数据集较少。本文提出ARKitTrack，这是一个全新的RGB-D跟踪数据集，涵盖由苹果iPhone和iPad上配备的消费级LiDAR扫描仪捕获的静态和动态场景。ARKitTrack包含300个RGB-D序列、455个目标以及总计229.7K帧视频帧。除边界框标注和帧级属性外，我们还为该数据集标注了123.9K个像素级目标掩码。此外，每帧的相机内参和相机姿态均已提供以供未来研究。为展示该数据集的潜在实用性，我们进一步提出一个统一的基线方法，用于框级和像素级跟踪，该方法将RGB特征与鸟瞰图表示相结合，以更好地探索跨模态3D几何信息。深入实证分析表明，ARKitTrack数据集能显著促进RGB-D跟踪，且所提出的基线方法性能优于现有最优方法。代码和数据集可从https://arkittrack.github.io获取。