We have recently observed the commercial roll-out of robotaxis in various countries. They are deployed within an operational design domain (ODD) on specific routes and environmental conditions, and are subject to continuous monitoring to regain control in safety-critical situations. Since ODDs typically cover urban areas, robotaxis must reliably detect vulnerable road users (VRUs) such as pedestrians, bicyclists, or e-scooter riders. To better handle such varied traffic situations, end-to-end AI, which directly compute vehicle control actions from multi-modal sensor data instead of only for perception, is on the rise. High quality data is needed for systematically training and evaluating such systems within their OOD. In this work, we propose PCICF, a framework to systematically identify and classify VRU situations to support ODD's incident analysis. We base our work on the existing synthetic dataset SMIRK, and enhance it by extending its single-pedestrian-only design into the MoreSMIRK dataset, a structured dictionary of multi-pedestrian crossing situations constructed systematically. We then use space-filling curves (SFCs) to transform multi-dimensional features of scenarios into characteristic patterns, which we match with corresponding entries in MoreSMIRK. We evaluate PCICF with the large real-world dataset PIE, which contains more than 150 manually annotated pedestrian crossing videos. We show that PCICF can successfully identify and classify complex pedestrian crossings, even when groups of pedestrians merge or split. By leveraging computationally efficient components like SFCs, PCICF has even potential to be used onboard of robotaxis for OOD detection for example. We share an open-source replication package for PCICF containing its algorithms, the complete MoreSMIRK dataset and dictionary, as well as our experiment results presented in: https://github.com/Claud1234/PCICF

翻译：近年来，我们观察到多国已开始商业部署机器人出租车。这些车辆在特定路线和环境条件下的运行设计域内运行，并受到持续监控，以便在安全关键情况下重新获得控制权。由于运行设计域通常覆盖城市区域，机器人出租车必须可靠地检测易受伤害的道路使用者，如行人、骑自行车者或电动滑板车骑行者。为了更好地处理此类多样的交通场景，端到端人工智能技术正在兴起，它直接从多模态传感器数据计算车辆控制动作，而不仅限于感知任务。为了在运行设计域内系统性地训练和评估此类系统，需要高质量的数据。在本研究中，我们提出PCICF框架，用于系统性地识别和分类易受伤害的道路使用者场景，以支持运行设计域的事件分析。我们的工作基于现有合成数据集SMIRK，并通过将其单一行人设计扩展为MoreSMIRK数据集进行增强——这是一个系统构建的多行人过街场景结构化词典。随后，我们使用空间填充曲线将场景的多维特征转换为特征模式，并与MoreSMIRK中的对应条目进行匹配。我们使用包含150多个手动标注行人过街视频的大型真实世界数据集PIE对PCICF进行评估。实验表明，PCICF能够成功识别和分类复杂的行人过街场景，即使在行人群体合并或分离的情况下也能有效工作。通过利用空间填充曲线等计算高效组件，PCICF甚至具备在机器人出租车上部署用于运行设计域检测的潜力。我们开源了PCICF的复现包，包含其算法、完整的MoreSMIRK数据集与词典，以及实验结果，详见：https://github.com/Claud1234/PCICF