Event cameras differ from conventional RGB cameras in that they produce asynchronous data sequences. While RGB cameras capture every frame at a fixed rate, event cameras only capture changes in the scene, resulting in sparse and asynchronous data output. Despite the fact that event data carries useful information that can be utilized in motion deblurring of RGB cameras, integrating event and image information remains a challenge. Recent state-of-the-art CNN-based deblurring solutions produce multiple 2-D event frames based on the accumulation of event data over a time period. In most of these techniques, however, the number of event frames is fixed and predefined, which reduces temporal resolution drastically, particularly for scenarios when fast-moving objects are present or when longer exposure times are required. It is also important to note that recent modern cameras (e.g., cameras in mobile phones) dynamically set the exposure time of the image, which presents an additional problem for networks developed for a fixed number of event frames. A Long Short-Term Memory (LSTM)-based event feature extraction module has been developed for addressing these challenges, which enables us to use a dynamically varying number of event frames. Using these modules, we constructed a state-of-the-art deblurring network, Deformable Convolutions and LSTM-based Flexible Event Frame Fusion Network (DLEFNet). It is particularly useful for scenarios in which exposure times vary depending on factors such as lighting conditions or the presence of fast-moving objects in the scene. It has been demonstrated through evaluation results that the proposed method can outperform the existing state-of-the-art networks for deblurring task in synthetic and real-world data sets.

翻译：事件相机与传统RGB相机不同，其产生异步数据序列。RGB相机以固定帧率捕捉每一帧图像，而事件相机仅捕捉场景中的变化，从而输出稀疏的异步数据。尽管事件数据携带有可用于RGB相机运动去模糊的有用信息，但如何融合事件与图像信息仍是一个挑战。近期基于CNN的最先进去模糊方法通过在一段时间内累积事件数据，生成多个二维事件帧。然而，在这些技术中，事件帧的数量通常是固定且预定义的，这显著降低了时间分辨率，尤其对于存在快速运动物体或需要更长曝光时间的场景而言。此外，值得注意的是，现代相机（例如手机摄像头）会动态设定图像的曝光时间，这为针对固定事件帧数量设计的网络带来了额外问题。为解决这些挑战，我们开发了一种基于长短期记忆（LSTM）的事件特征提取模块，使其能够使用动态变化的事件帧数量。利用这些模块，我们构建了最先进的去模糊网络——基于可变形卷积与LSTM的灵活事件帧融合网络（DLEFNet）。该网络特别适用于曝光时间因光照条件或场景中存在快速运动物体等因素而变化的场景。评估结果表明，该方法在合成数据集与真实世界数据集上的去模糊任务中均优于现有最先进网络。