Visual Odometry (VO) is one of the fundamental tasks in computer vision for robotics. However, its performance is deeply affected by High Dynamic Range (HDR) scenes, omnipresent outdoor. While new Automatic-Exposure (AE) approaches to mitigate this have appeared, their comparison in a reproducible manner is problematic. This stems from the fact that the behavior of AE depends on the environment, and it affects the image acquisition process. Consequently, AE has traditionally only been benchmarked in an online manner, making the experiments non-reproducible. To solve this, we propose a new methodology based on an emulator that can generate images at any exposure time. It leverages BorealHDR, a unique multi-exposure stereo dataset collected over 10 km, on 55 trajectories with challenging illumination conditions. Moreover, it includes lidar-inertial-based global maps with pose estimation for each image frame as well as Global Navigation Satellite System (GNSS) data, for comparison. We show that using these images acquired at different exposure times, we can emulate realistic images, keeping a Root-Mean-Square Error (RMSE) below 1.78 % compared to ground truth images. To demonstrate the practicality of our approach for offline benchmarking, we compared three state-of-the-art AE algorithms on key elements of Visual Simultaneous Localization And Mapping (VSLAM) pipeline, against four baselines. Consequently, reproducible evaluation of AE is now possible, speeding up the development of future approaches. Our code and dataset are available online at this link: https://github.com/norlab-ulaval/BorealHDR

翻译：视觉里程计（VO）是机器人领域计算机视觉的基础任务之一，但其性能深受户外普遍存在的高动态范围（HDR）场景影响。尽管近期出现了缓解该问题的自动曝光（AE）新方法，但以可复现方式对其进行对比仍存在困难。这源于AE的行为依赖于环境，且会影响图像采集过程。因此，AE传统上仅能以在线方式进行基准测试，导致实验不可复现。为解决该问题，我们提出了一种基于仿真器的新方法，该仿真器可生成任意曝光时间下的图像。方法基于BorealHDR——一个独特的、在55条轨迹上历经10公里采集的多曝光立体数据集，涵盖了具有挑战性的光照条件。此外，该数据集包含基于激光雷达-惯导的全局地图（含每幅图像帧的位姿估计）及全球导航卫星系统（GNSS）数据以供对比。我们证明，利用以不同曝光时间采集的图像，可仿真出真实图像，其均方根误差（RMSE）相较于真实图像低于1.78%。为展示该方法在离线基准测试中的实用性，我们针对视觉同步定位与地图构建（VSLAM）流水线的关键要素，将三种最先进的AE算法与四种基线进行了对比。由此，AE的可复现评估成为可能，从而加速未来方法的发展。我们的代码和数据集可通过以下链接获取：https://github.com/norlab-ulaval/BorealHDR