The high-dimensional nature of the 4-D light field (LF) poses great challenges in achieving efficient and effective feature embedding, that severely impacts the performance of downstream tasks. To tackle this crucial issue, in contrast to existing methods with empirically-designed architectures, we propose a probabilistic-based feature embedding (PFE), which learns a feature embedding architecture by assembling various low-dimensional convolution patterns in a probability space for fully capturing spatial-angular information. Building upon the proposed PFE, we then leverage the intrinsic linear imaging model of the coded aperture camera to construct a cycle-consistent 4-D LF reconstruction network from coded measurements. Moreover, we incorporate PFE into an iterative optimization framework for 4-D LF denoising. Our extensive experiments demonstrate the significant superiority of our methods on both real-world and synthetic 4-D LF images, both quantitatively and qualitatively, when compared with state-of-the-art methods. The source code will be publicly available at https://github.com/lyuxianqiang/LFCA-CR-NET.
翻译:四维光场的高维特性给高效且有效的特征嵌入带来了巨大挑战,严重影响了后续任务的性能。为解决这一关键问题,与现有基于经验设计架构的方法不同,我们提出了一种基于概率的特征嵌入(PFE),通过在概率空间中组装多种低维卷积模式来学习特征嵌入架构,以充分捕获空间-角度信息。基于所提出的PFE,我们进一步利用编码孔径相机的固有线性成像模型,构建了从编码测量中重建四维光场的循环一致性网络。此外,我们将PFE融入迭代优化框架中,用于四维光场去噪。大量实验表明,与现有最先进方法相比,我们的方法在真实和合成的四维光场图像上,无论是在定量还是定性评估方面均具有显著优越性。源代码将在https://github.com/lyuxianqiang/LFCA-CR-NET 公开提供。