Large-scale driving datasets such as Waymo Open Dataset and nuScenes substantially accelerate autonomous driving research, especially for perception tasks such as 3D detection and trajectory forecasting. Since the driving logs in these datasets contain HD maps and detailed object annotations which accurately reflect the real-world complexity of traffic behaviors, we can harvest a massive number of complex traffic scenarios and recreate their digital twins in simulation. Compared to the hand-crafted scenarios often used in existing simulators, data-driven scenarios collected from the real world can facilitate many research opportunities in machine learning and autonomous driving. In this work, we present ScenarioNet, an open-source platform for large-scale traffic scenario modeling and simulation. ScenarioNet defines a unified scenario description format and collects a large-scale repository of real-world traffic scenarios from the heterogeneous data in various driving datasets including Waymo, nuScenes, Lyft L5, and nuPlan datasets. These scenarios can be further replayed and interacted with in multiple views from Bird-Eye-View layout to realistic 3D rendering in MetaDrive simulator. This provides a benchmark for evaluating the safety of autonomous driving stacks in simulation before their real-world deployment. We further demonstrate the strengths of ScenarioNet on large-scale scenario generation, imitation learning, and reinforcement learning in both single-agent and multi-agent settings. Code, demo videos, and website are available at https://github.com/metadriverse.github.io/scenarionet.

翻译：诸如Waymo Open Dataset和nuScenes等大规模驾驶数据集极大地加速了自动驾驶研究，尤其是针对3D检测和轨迹预测等感知任务。由于这些数据集中的驾驶日志包含高清地图和详细的目标标注，能够准确反映真实世界中交通行为的复杂性，因此我们可以从中获取海量复杂交通场景，并在仿真中重建其数字孪生。与现有仿真器中常使用的手工构建场景相比，从真实世界采集的数据驱动场景可为机器学习和自动驾驶领域的研究提供更多机遇。本工作中，我们提出ScenarioNet——一个面向大规模交通场景建模与仿真的开源平台。ScenarioNet定义了统一的场景描述格式，并从Waymo、nuScenes、Lyft L5和nuPlan等多个驾驶数据集的异构数据中收集了大规模真实交通场景库。这些场景可在MetaDrive仿真器中，通过鸟瞰视图到逼真3D渲染的多视角进行回放与交互，从而为自动驾驶堆栈在真实世界部署前的安全性评估提供基准。我们进一步展示了ScenarioNet在大规模场景生成、模仿学习以及单智能体和多智能体设置下的强化学习等方面的优势。代码、演示视频及网站详见https://github.com/metadriverse.github.io/scenarionet。