Scenario-based testing is a promising approach to solve the challenge of proving the safe behavior of vehicles equipped with automated driving systems. Since an infinite number of concrete scenarios can theoretically occur in real-world road traffic, the extraction of scenarios relevant in terms of the safety-related behavior of these systems is a key aspect for their successful verification and validation. Therefore, a method for extracting multimodal urban traffic scenarios from naturalistic road traffic data in an unsupervised manner, minimizing the amount of (potentially biased) prior expert knowledge, is proposed. Rather than an (elaborate) rule-based assignment by extracting concrete scenarios into predefined functional scenarios, the presented method deploys an unsupervised machine learning pipeline. The approach allows exploring the unknown nature of the data and their interpretation as test scenarios that experts could not have anticipated. The method is evaluated for naturalistic road traffic data at urban intersections from the inD and the Silicon Valley Intersections datasets. For this purpose, it is analyzed with which clustering approach (K-Means, hierarchical clustering, and DBSCAN) the scenario extraction method performs best (referring to an elaborate rule-based implementation). Subsequently, using hierarchical clustering the results show both a jump in overall accuracy of around 20% when moving from 4 to 5 clusters and a saturation effect starting at 41 clusters with an overall accuracy of 84%. These observations can be a valuable contribution in the context of the trade-off between the number of functional scenarios (i.e., clustering accuracy) and testing effort. Possible reasons for the observed accuracy variations of different clusters, each with a fixed total number of given clusters, are discussed.
翻译:基于场景的测试是解决配备自动驾驶系统的车辆安全行为验证挑战的一种有前景的方法。由于现实道路交通中理论上可能发生的具体场景数量无限,提取与这些系统安全相关行为相关的场景成为其成功验证与确认的关键。为此,提出一种从自然道路交通数据中以无监督方式提取多模态城市交通场景的方法,以最小化(可能带有偏见的)先验专家知识的干扰。所提方法并非采用(复杂的)基于规则的分配方式将具体场景映射到预定义功能场景,而是部署无监督机器学习流水线。该方法能够探索数据的未知特性及其作为测试场景的解释,而这些场景可能是专家无法预见的。我们使用inD和Silicon Valley Intersections数据集中城市交叉路口的自然道路交通数据对该方法进行评估。为此,分析了场景提取方法在何种聚类方法(K-Means、层次聚类和DBSCAN)下表现最佳(以复杂的基于规则的实现为参照)。随后,采用层次聚类的结果显示,当从4个聚类增加到5个时,整体准确率提升约20%,而当聚类数达到41个时,准确率达到84%并出现饱和效应。这些观察对于功能场景数量(即聚类准确率)与测试工作之间的权衡具有重要参考价值。最后,讨论了在给定总聚类数不变的情况下各聚类准确率变化的可能原因。