In the life cycle of highly automated systems operating in an open and dynamic environment, the ability to adjust to emerging challenges is crucial. For systems integrating data-driven AI-based components, rapid responses to deployment issues require fast access to related data for testing and reconfiguration. In the context of automated driving, this especially applies to road obstacles that were not included in the training data, commonly referred to as out-of-distribution (OoD) road obstacles. Given the availability of large uncurated recordings of driving scenes, a pragmatic approach is to query a database to retrieve similar scenarios featuring the same safety concerns due to OoD road obstacles. In this work, we extend beyond identifying OoD road obstacles in video streams and offer a comprehensive approach to extract sequences of OoD road obstacles using text queries, thereby proposing a way of curating a collection of OoD data for subsequent analysis. Our proposed method leverages the recent advances in OoD segmentation and multi-modal foundation models to identify and efficiently extract safety-relevant scenes from unlabeled videos. We present a first approach for the novel task of text-based OoD object retrieval, which addresses the question ''Have we ever encountered this before?''.

翻译：在开放动态环境中运行的高度自动化系统的生命周期中，适应新挑战的能力至关重要。对于集成基于数据驱动的人工智能组件的系统，快速响应部署问题需要及时访问相关数据以进行测试和重新配置。在自动驾驶背景下，这尤其适用于未包含在训练数据中的道路障碍物，通常称为分布外（OoD）道路障碍物。鉴于大量未整理的驾驶场景记录的存在，一种实用的方法是查询数据库以检索涉及相同OoD道路障碍物安全问题的相似场景。在本工作中，我们不仅限于识别视频流中的OoD道路障碍物，还提出了一种综合方法，通过文本查询提取OoD道路障碍物序列，从而为后续分析提供一种策划OoD数据集合的方式。我们的方法利用了OoD分割和多模态基础模型的最新进展，以识别并高效提取来自未标记视频的安全相关场景。我们针对基于文本的OoD对象检索这一新任务提出了首个方法，该方法旨在回答“我们是否曾遇见过这个？”这一问题。