Event cameras are innovative neuromorphic sensors that asynchronously capture the scene dynamics. Due to the event-triggering mechanism, such cameras record event streams with much shorter response latency and higher intensity sensitivity compared to conventional cameras. On the basis of these features, previous works have attempted to reconstruct high dynamic range (HDR) videos from events, but have either suffered from unrealistic artifacts or failed to provide sufficiently high frame rates. In this paper, we present a recurrent convolutional neural network that reconstruct high-speed HDR videos from event sequences, with a key frame guidance to prevent potential error accumulation caused by the sparse event data. Additionally, to address the problem of severely limited real dataset, we develop a new optical system to collect a real-world dataset with paired high-speed HDR videos and event streams, facilitating future research in this field. Our dataset provides the first real paired dataset for event-to-HDR reconstruction, avoiding potential inaccuracies from simulation strategies. Experimental results demonstrate that our method can generate high-quality, high-speed HDR videos. We further explore the potential of our work in cross-camera reconstruction and downstream computer vision tasks, including object detection, panoramic segmentation, optical flow estimation, and monocular depth estimation under HDR scenarios.

翻译：事件相机是一种创新的神经形态传感器，能够异步捕捉场景动态。得益于事件触发机制，此类相机记录的事件流相比传统相机具有更短的响应延迟和更高的强度灵敏度。基于这些特性，先前的研究尝试从事件重建高动态范围（HDR）视频，但往往存在不真实的伪影或无法提供足够高的帧率。本文提出一种循环卷积神经网络，可从事件序列重建高速HDR视频，并引入关键帧引导机制以避免稀疏事件数据可能导致的误差累积。此外，针对真实数据集严重受限的问题，我们开发了一套新型光学系统以采集包含配对高速HDR视频与事件流的真实世界数据集，为未来该领域的研究提供支持。我们的数据集首次为事件到HDR重建任务提供了真实配对数据，避免了仿真策略可能引入的误差。实验结果表明，我们的方法能够生成高质量的高速HDR视频。我们进一步探索了该工作在多相机重建及下游计算机视觉任务中的潜力，包括HDR场景下的目标检测、全景分割、光流估计和单目深度估计。