Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/.

翻译：光场（LF）相机同时记录光线的强度和方向，将三维场景编码为四维光场图像。近年来，针对各种光场图像处理任务，人们提出了许多卷积神经网络（CNN）。然而，由于空间信息和角度信息以不同的视差高度交织，CNN难以有效处理光场图像。本文提出了一种通用机制来解耦这些耦合信息以用于光场图像处理。具体来说，我们首先设计了一类领域特定卷积来从不同维度解耦光场，然后通过设计任务特定模块来利用这些解耦特征。我们的解耦机制能够很好地融入光场结构先验，并有效处理四维光场数据。基于所提出的机制，我们开发了三种网络（即DistgSSR、DistgASR和DistgDisp），分别用于空间超分辨率、角度超分辨率和视差估计。实验结果表明，我们的网络在这三个任务上均达到了最先进的性能，这证明了我们解耦机制的有效性、高效性和通用性。项目页面：https://yingqianwang.github.io/DistgLF/。