Visual Odometry (VO) is one of the fundamental tasks in computer vision for robotics. However, its performance is deeply affected by High Dynamic Range (HDR) scenes, omnipresent outdoor. While new Automatic-Exposure (AE) approaches to mitigate this have appeared, their comparison in a reproducible manner is problematic. This stems from the fact that the behavior of AE depends on the environment, and it affects the image acquisition process. Consequently, AE has traditionally only been benchmarked in an online manner, making the experiments non-reproducible. To solve this, we propose a new methodology based on an emulator that can generate images at any exposure time. It leverages BorealHDR, a unique multi-exposure stereo dataset collected over 10 km, on 55 trajectories with challenging illumination conditions. Moreover, it includes lidar-inertial-based global maps with pose estimation for each image frame as well as Global Navigation Satellite System (GNSS) data, for comparison. We show that using these images acquired at different exposure times, we can emulate realistic images, keeping a Root-Mean-Square Error (RMSE) below 1.78 % compared to ground truth images. To demonstrate the practicality of our approach for offline benchmarking, we compared three state-of-the-art AE algorithms on key elements of Visual Simultaneous Localization And Mapping (VSLAM) pipeline, against four baselines. Consequently, reproducible evaluation of AE is now possible, speeding up the development of future approaches. Our code and dataset are available online at this link: https://github.com/norlab-ulaval/BorealHDR

翻译：视觉里程计（VO）是机器人领域计算机视觉中的基础任务之一，但其性能深受户外普遍存在的高动态范围（HDR）场景影响。尽管近期出现了缓解此问题的新型自动曝光（AE）方法，但以可复现方式对其进行比较存在困难。这源于AE行为依赖于环境且影响图像采集过程。因此，AE传统上仅能通过在线方式进行基准测试，导致实验无法复现。为解决此问题，我们提出了一种基于仿真器的新方法，该仿真器可生成任意曝光时间的图像。该方法利用BorealHDR——一个在55条轨迹、全长10公里的具有挑战性光照条件下采集的独特多曝光立体数据集。此外，该数据集包含基于激光雷达-惯性系统的全局地图（含每帧图像的位姿估计）及全球导航卫星系统（GNSS）数据，便于比较。我们证明，利用不同曝光时间采集的图像，可仿真出真实图像，其与真实图像相比均方根误差（RMSE）低于1.78%。为展示该方法在离线基准测试中的实用性，我们针对视觉同步定位与建图（VSLAM）流程的关键环节，将三种最先进的AE算法与四种基线方法进行了比较。由此，AE的可复现评估成为可能，从而加速未来方法的发展。我们的代码和数据集发布于以下链接：https://github.com/norlab-ulaval/BorealHDR