Video snapshot compressive imaging (SCI) uses a two-dimensional detector to capture consecutive video frames during a single exposure time. Following this, an efficient reconstruction algorithm needs to be designed to reconstruct the desired video frames. Although recent deep learning-based state-of-the-art (SOTA) reconstruction algorithms have achieved good results in most tasks, they still face the following challenges due to excessive model complexity and GPU memory limitations: 1) these models need high computational cost, and 2) they are usually unable to reconstruct large-scale video frames at high compression ratios. To address these issues, we develop an efficient network for video SCI by using dense connections and space-time factorization mechanism within a single residual block, dubbed EfficientSCI. The EfficientSCI network can well establish spatial-temporal correlation by using convolution in the spatial domain and Transformer in the temporal domain, respectively. We are the first time to show that an UHD color video with high compression ratio can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 32 dB. Extensive results on both simulation and real data show that our method significantly outperforms all previous SOTA algorithms with better real-time performance. The code is at https://github.com/ucaswangls/EfficientSCI.git.

翻译：视频快照压缩成像（SCI）利用二维探测器在单次曝光时间内捕获连续视频帧，随后需设计高效重建算法恢复目标视频帧。尽管基于深度学习的最新（SOTA）重建算法在多数任务中取得了良好效果，但由于模型复杂度高及GPU内存限制，仍面临以下挑战：1）模型计算成本高昂，2）难以在高压缩比下重建大规模视频帧。针对这些问题，我们提出一种高效视频SCI网络——EfficientSCI，其通过单个残差块内的密集连接与时空分解机制构建时空关联（空间域采用卷积，时间域采用Transformer）。本研究首次证明，使用单个端到端深度学习模型即可从快照二维测量中重建高压缩比超高清（UHD）彩色视频，峰值信噪比（PSNR）超过32 dB。在仿真与真实数据上的大量实验表明，本方法在实时性能上全面超越所有先前SOTA算法。代码见：https://github.com/ucaswangls/EfficientSCI.git。