We present 4DLidarOpen, a large-scale open multi-modal dataset for autonomous driving, centered on 4D frequency-modulated continuous-wave (FMCW) Lidar sensing. Unlike conventional time-of-flight Lidar datasets that mainly provide geometric measurements, 4DLidarOpen includes point-wise radial velocity measurements from a forward-facing 4D FMCW Lidar, together with multiple Lidars of different types, including rotating, solid-state, and blind-spot variants, surround-view cameras, and 6-DOF ego-vehicle poses. The dataset was collected in complex urban environments in Beijing and covers dense pedestrian interactions, congested traffic, high-speed driving, and unprotected maneuvers. 4DLidarOpen provides synchronized multi-sensor data and 3D bounding-box annotations with persistent track IDs across five object categories. A hybrid annotation strategy is adopted, where large-scale auto-labeled data support scalable training and human experts refine annotations for the human-annotated training and validation sets. Based on this dataset, we establish benchmarks for 3D object detection, birds-eye view (BEV) segmentation and flow prediction, and motion forecasting with planning. Extensive experiments show that direct velocity measurements from 4D FMCW Lidar provide complementary motion cues for dynamic-scene understanding. Compared with geometric-only sensing, the velocity-aware representation improves motion-related perception and downstream forecasting and planning, especially in scenarios involving vulnerable road users and fast-moving objects. These results indicate that 4D FMCW Lidar is a promising sensing modality for motion-aware autonomous driving. The dataset and evaluation toolkit are publicly released to support research on 4D scene understanding, multi-Lidar fusion, and velocity-aware perception and planning.

翻译：本文介绍了4DLidarOpen，一个面向自动驾驶的大规模开源多模态数据集，其核心为4D调频连续波（FMCW）激光雷达传感。与传统飞行时间激光雷达数据集主要提供几何测量不同，4DLidarOpen包含来自前向4D FMCW激光雷达的点云径向速度测量数据，以及多种不同类型的激光雷达（包括旋转式、固态式和盲区变体）、环视摄像头和六自由度自车位姿数据。该数据集在北京复杂城区环境中采集，覆盖密集行人交互、拥堵交通、高速行驶和无保护机动等场景。4DLidarOpen提供同步多传感器数据及带有持续轨迹ID的5个目标类别的3D边界框标注。我们采用混合标注策略：大规模自动标注数据支持可扩展训练，而人工专家标注则用于优化人工训练集和验证集。基于该数据集，我们建立了包括3D目标检测、鸟瞰图（BEV）分割、光流预测以及运动预测与规划的基准测试。大量实验表明，4D FMCW激光雷达的直接速度测量为动态场景理解提供了互补的运动线索。与纯几何传感相比，速度感知表征能提升运动相关感知及下游预测与规划性能，尤其在涉及弱势道路使用者和快速移动目标的场景中。这些结果证明，4D FMCW激光雷达是面向运动感知自动驾驶的一种极具前景的传感模态。数据集与评估工具包已开源发布，以支持4D场景理解、多激光雷达融合及速度感知与规划的研究。