Autonomous vehicles (AV) are expected to reshape future transportation systems, and decision-making is one of the critical modules toward high-level automated driving. To overcome those complicated scenarios that rule-based methods could not cope with well, data-driven decision-making approaches have aroused more and more focus. The datasets to be used in developing data-driven methods dramatically influences the performance of decision-making, hence it is necessary to have a comprehensive insight into the existing datasets. From the aspects of collection sources, driving data can be divided into vehicle, environment, and driver related data. This study compares the state-of-the-art datasets of these three categories and summarizes their features including sensors used, annotation, and driving scenarios. Based on the characteristics of the datasets, this survey also concludes the potential applications of datasets on various aspects of AV decision-making, assisting researchers to find appropriate ones to support their own research. The future trends of AV dataset development are summarized.

翻译：自动驾驶车辆有望重塑未来交通系统，而决策是实现高级自动驾驶的关键模块之一。为克服基于规则的方法难以应对的复杂场景，数据驱动型决策方法日益受到关注。用于开发数据驱动方法的数据集会显著影响决策性能，因此有必要全面洞察现有数据集。从采集来源角度出发，驾驶数据可分为车辆相关数据、环境相关数据及驾驶员相关数据三类。本研究对比了这三类中具有代表性的最新数据集，并从传感器配置、标注方式及驾驶场景等维度总结其特性。基于数据集特征，本文还归纳了数据集在自动驾驶车辆决策各环节中的潜在应用场景，有助于研究者选取适合自身研究的数据集。最后总结了自动驾驶车辆数据集的发展趋势。