Light field cameras have a wide range of uses due to their ability to simultaneously record light intensity and direction. The angular resolution of light fields is important for downstream tasks such as depth estimation, yet is often difficult to improve due to hardware limitations. Conventional methods tend to perform poorly against the challenge of large disparity in sparse light fields, while general CNNs have difficulty extracting spatial and angular features coupled together in 4D light fields. The light field disentangling mechanism transforms the 4D light field into 2D image format, which is more favorable for CNN for feature extraction. In this paper, we propose a Deep Disentangling Mechanism, which inherits the principle of the light field disentangling mechanism and further develops the design of the feature extractor and adds advanced network structure. We design a light-field reconstruction network (i.e., DDASR) on the basis of the Deep Disentangling Mechanism, and achieve SOTA performance in the experiments. In addition, we design a Block Traversal Angular Super-Resolution Strategy for the practical application of depth estimation enhancement where the input views is often higher than 2x2 in the experiments resulting in a high memory usage, which can reduce the memory usage while having a better reconstruction performance.
翻译:光场相机因其能够同时记录光线的强度和方向而具有广泛的应用。光场的角分辨率对于深度估计等下游任务至关重要,但由于硬件限制,往往难以提升。传统方法在面对稀疏光场中的大视差挑战时表现不佳,而通用CNN难以提取4D光场中耦合在一起的空间和角度特征。光场解耦机制将4D光场转换为更有利于CNN进行特征提取的2D图像格式。本文提出了一种深度解耦机制,该机制继承了光场解耦原理,进一步改进了特征提取器的设计,并引入了先进的网络结构。我们基于深度解耦机制设计了光场重建网络(即DDASR),并在实验中取得了SOTA性能。此外,针对深度估计增强的实际应用,我们设计了块遍历角度超分辨率策略,当输入视角通常高于2x2时,实验中的内存使用量较高,该策略能够降低内存使用量,同时具有更好的重建性能。