With the rise of Extended Reality (XR) technology, there is a growing need for real-time light field generation from sparse view inputs. Existing methods can be classified into offline techniques, which can generate high-quality novel views but at the cost of long inference/training time, and online methods, which either lack generalizability or produce unsatisfactory results. However, we have observed that the intrinsic sparse manifold of Multi-plane Images (MPI) enables a significant acceleration of light field generation while maintaining rendering quality. Based on this insight, we introduce EffLiFe, a novel light field optimization method, which leverages the proposed Hierarchical Sparse Gradient Descent (HSGD) to produce high-quality light fields from sparse view images in real time. Technically, the coarse MPI of a scene is first generated using a 3D CNN, and it is further sparsely optimized by focusing only on important MPI gradients in a few iterations. Nevertheless, relying solely on optimization can lead to artifacts at occlusion boundaries. Therefore, we propose an occlusion-aware iterative refinement module that removes visual artifacts in occluded regions by iteratively filtering the input. Extensive experiments demonstrate that our method achieves comparable visual quality while being 100x faster on average than state-of-the-art offline methods and delivering better performance (about 2 dB higher in PSNR) compared to other online approaches.

翻译：随着扩展现实（XR）技术的兴起，从稀疏视角输入生成实时光场的需求日益增长。现有方法可分为离线技术和在线方法：离线技术虽能生成高质量的新视角，但推理/训练时间较长；在线方法则要么缺乏泛化能力，要么产生不理想的结果。然而，我们发现多平面图像（MPI）固有的稀疏流形结构，能够在维持渲染质量的同时显著加速光场生成。基于这一洞察，我们提出了EffLiFe——一种新颖的光场优化方法，该方法利用所提出的分层稀疏梯度下降（HSGD）从稀疏视角图像实时生成高质量光场。在技术实现上，首先使用3D CNN生成场景的粗略MPI，然后通过仅关注少量迭代中的关键MPI梯度进行稀疏优化。然而，仅依赖优化会导致遮挡边界出现伪影。因此，我们提出了一种遮挡感知的迭代细化模块，通过迭代滤波输入来消除遮挡区域中的视觉伪影。大量实验表明，我们的方法在实现可比视觉质量的同时，平均速度比最先进的离线方法快100倍，并且相比其他在线方法实现了更优性能（PSNR高出约2 dB）。