Autonomous vehicles (AV) are expected to reshape future transportation systems, and decision-making is one of the critical modules toward high-level automated driving. To overcome those complicated scenarios that rule-based methods could not cope with well, data-driven decision-making approaches have aroused more and more focus. The datasets to be used in developing data-driven methods dramatically influences the performance of decision-making, hence it is necessary to have a comprehensive insight into the existing datasets. From the aspects of collection sources, driving data can be divided into vehicle, environment, and driver related data. This study compares the state-of-the-art datasets of these three categories and summarizes their features including sensors used, annotation, and driving scenarios. Based on the characteristics of the datasets, this survey also concludes the potential applications of datasets on various aspects of AV decision-making, assisting researchers to find appropriate ones to support their own research. The future trends of AV dataset development are summarized.

翻译：自主驾驶车辆有望重塑未来交通系统，而决策是实现高级自动驾驶的关键模块之一。为克服基于规则的方法难以妥善处理的复杂场景，数据驱动的决策方法日益受到关注。用于开发数据驱动方法的数据集对决策性能具有显著影响，因此全面了解现有数据集十分必要。从采集来源角度，驾驶数据可分为车辆、环境和驾驶员三类数据。本研究对比了这三类前沿数据集，并总结了其特性，包括所使用的传感器、标注方式及驾驶场景。基于数据集特征，本综述还归纳了数据集在自主驾驶车辆决策各环节中的潜在应用，以协助研究人员选取适合自身研究的数据集。最后总结了自主驾驶车辆数据集开发的未来趋势。