Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/.

翻译：光场（LF）相机同时记录光线的强度和方向，并将三维场景编码为四维光场图像。近年来，研究人员针对各种光场图像处理任务提出了多种卷积神经网络（CNN）。然而，由于空间信息和角度信息在不同视差下高度耦合，CNN难以有效处理光场图像。本文提出了一种通用机制来解耦这些耦合信息以进行光场图像处理。具体而言，我们首先设计了一类域特定卷积，从不同维度解耦光场；随后通过设计任务特定模块来利用这些解耦特征。我们的解耦机制能够很好地融入光场结构先验，并有效处理四维光场数据。基于该机制，我们开发了三种网络（即DistgSSR、DistgASR和DistgDisp），分别用于空间超分辨率、角度超分辨率和视差估计。实验结果表明，我们的网络在这三项任务上均达到了最先进的性能，充分验证了解耦机制的有效性、高效性和通用性。项目页面：https://yingqianwang.github.io/DistgLF/。