The high-dimensional nature of the 4-D light field (LF) poses great challenges in achieving efficient and effective feature embedding, that severely impacts the performance of downstream tasks. To tackle this crucial issue, in contrast to existing methods with empirically-designed architectures, we propose a probabilistic-based feature embedding (PFE), which learns a feature embedding architecture by assembling various low-dimensional convolution patterns in a probability space for fully capturing spatial-angular information. Building upon the proposed PFE, we then leverage the intrinsic linear imaging model of the coded aperture camera to construct a cycle-consistent 4-D LF reconstruction network from coded measurements. Moreover, we incorporate PFE into an iterative optimization framework for 4-D LF denoising. Our extensive experiments demonstrate the significant superiority of our methods on both real-world and synthetic 4-D LF images, both quantitatively and qualitatively, when compared with state-of-the-art methods. The source code will be publicly available at https://github.com/lyuxianqiang/LFCA-CR-NET.
翻译:四维光场的高维特性对实现高效且有效的特征嵌入提出了巨大挑战,严重影响了下游任务的性能。为解决这一关键问题,与现有依赖经验设计架构的方法不同,我们提出了一种基于概率的特征嵌入方法(PFE),该方法通过在概率空间中组合各种低维卷积模式来学习特征嵌入架构,以充分捕获空间-角度信息。在此基础上,我们进一步利用编码孔径相机的固有线性成像模型,构建了一个从编码测量中恢复四维光场的循环一致性重建网络。此外,我们将PFE集成到迭代优化框架中,用于四维光场去噪。大量的实验表明,与最先进的方法相比,我们的方法在真实和合成四维光场图像上均展现出显著的定量和定性优势。源代码将在https://github.com/lyuxianqiang/LFCA-CR-NET上公开。