As a neuromorphic sensor with high temporal resolution, spike camera can generate continuous binary spike streams to capture per-pixel light intensity. We can use reconstruction methods to restore scene details in high-speed scenarios. However, due to limited information in spike streams, low-light scenes are difficult to effectively reconstruct. In this paper, we propose a bidirectional recurrent-based reconstruction framework, including a Light-Robust Representation (LR-Rep) and a fusion module, to better handle such extreme conditions. LR-Rep is designed to aggregate temporal information in spike streams, and a fusion module is utilized to extract temporal features. Additionally, we have developed a reconstruction benchmark for high-speed low-light scenes. Light sources in the scenes are carefully aligned to real-world conditions. Experimental results demonstrate the superiority of our method, which also generalizes well to real spike streams. Related codes and proposed datasets will be released after publication.
翻译:作为一种具有高时间分辨率的神经形态传感器,脉冲相机能够生成连续的二进制脉冲流以捕获每个像素的光强度。我们可以利用重建方法恢复高速场景中的细节信息。然而,由于脉冲流中包含的信息有限,低光照场景难以有效重建。本文提出一种基于双向循环的重建框架,包括光鲁棒表示模块(LR-Rep)和融合模块,以更好地处理此类极端条件。LR-Rep用于聚合脉冲流中的时间信息,融合模块则用于提取时序特征。此外,我们建立了高速低光照场景的重建基准,场景中的光源经过精心校准以符合真实环境。实验结果表明,本方法具有优越性,并能很好地泛化到实际脉冲流中。相关代码与提出的数据集将在发表后公开。