Conventional cameras employed in autonomous vehicle (AV) systems support many perception tasks, but are challenged by low-light or high dynamic range scenes, adverse weather, and fast motion. Novel sensors, such as event and thermal cameras, offer capabilities with the potential to address these scenarios, but they remain to be fully exploited. This paper introduces the Novel Sensors for Autonomous Vehicle Perception (NSAVP) dataset to facilitate future research on this topic. The dataset was captured with a platform including stereo event, thermal, monochrome, and RGB cameras as well as a high precision navigation system providing ground truth poses. The data was collected by repeatedly driving two ~8 km routes and includes varied lighting conditions and opposing viewpoint perspectives. We provide benchmarking experiments on the task of place recognition to demonstrate challenges and opportunities for novel sensors to enhance critical AV perception tasks. To our knowledge, the NSAVP dataset is the first to include stereo thermal cameras together with stereo event and monochrome cameras. The dataset and supporting software suite is available at: https://umautobots.github.io/nsavp

翻译：摘要：传统摄像头在自动驾驶车辆系统中支持诸多感知任务，但在低光照、高动态范围场景、恶劣天气及快速运动条件下面临挑战。事件相机和热成像相机等新型传感器具备应对这些场景的潜力，但尚未被充分利用。本文提出“面向自动驾驶感知的新型传感器数据集（NSAVP）”，以促进相关领域研究。该数据集通过配备立体事件相机、热成像相机、单色相机及RGB相机的采集平台获得，并集成高精度导航系统提供真值位姿。数据通过沿两条约8公里路线重复行驶采集，涵盖多样光照条件及对向视角。我们针对地点识别任务开展基准实验，以揭示新型传感器在增强关键自动驾驶感知任务中的挑战与机遇。据我们所知，NSAVP数据集是首个同时包含立体热成像相机、立体事件相机及单色相机的公开数据集。数据集及配套软件工具包可通过以下链接获取：https://umautobots.github.io/nsavp