Event cameras are emerging imaging technology that offers advantages over conventional frame-based imaging sensors in dynamic range and sensing speed. Complementing the rich texture and color perception of traditional image frames, the hybrid camera system of event and frame-based cameras enables high-performance imaging. With the assistance of event cameras, high-quality image/video enhancement methods make it possible to break the limits of traditional frame-based cameras, especially exposure time, resolution, dynamic range, and frame rate limits. This paper focuses on five event-aided image and video enhancement tasks (i.e., event-based video reconstruction, event-aided high frame rate video reconstruction, image deblurring, image super-resolution, and high dynamic range image reconstruction), provides an analysis of the effects of different event properties, a real-captured and ground truth labeled benchmark dataset, a unified benchmarking of state-of-the-art methods, and an evaluation for two mainstream event simulators. In detail, this paper collects a real-captured evaluation dataset EventAid for five event-aided image/video enhancement tasks, by using "Event-RGB" multi-camera hybrid system, taking into account scene diversity and spatiotemporal synchronization. We further perform quantitative and visual comparisons for state-of-the-art algorithms, provide a controlled experiment to analyze the performance limit of event-aided image deblurring methods, and discuss open problems to inspire future research.

翻译：事件相机是一种新兴的成像技术，在动态范围和感知速度方面优于传统帧式成像传感器。为弥补传统图像帧在丰富纹理和色彩感知上的不足，事件相机与帧式相机的混合相机系统实现了高性能成像。借助事件相机的辅助，高质量图像/视频增强方法有望突破传统帧式相机的限制，特别是在曝光时间、分辨率、动态范围和帧率方面。本文聚焦于五项事件辅助图像与视频增强任务（即基于事件的视频重建、事件辅助高帧率视频重建、图像去模糊、图像超分辨率及高动态范围图像重建），分析了不同事件属性的影响，构建了包含真实采集数据及真实标注的基准数据集，对现有最优方法进行了统一基准测试，并评估了两种主流事件模拟器。具体而言，本文利用“事件-彩色”多相机混合系统，综合考虑场景多样性与时空同步性，为五项事件辅助图像/视频增强任务采集了真实场景评估数据集EventAid。进一步，我们对现有最优算法进行了定量与视觉对比，通过受控实验分析了事件辅助图像去模糊方法的性能极限，并探讨了开放性问题以启发未来研究。