With the rise of Extended Reality (XR) technology, there is a growing need for real-time light field generation from sparse view inputs. Existing methods can be classified into offline techniques, which can generate high-quality novel views but at the cost of long inference/training time, and online methods, which either lack generalizability or produce unsatisfactory results. However, we have observed that the intrinsic sparse manifold of Multi-plane Images (MPI) enables a significant acceleration of light field generation while maintaining rendering quality. Based on this insight, we introduce EffLiFe, a novel light field optimization method, which leverages the proposed Hierarchical Sparse Gradient Descent (HSGD) to produce high-quality light fields from sparse view images in real time. Technically, the coarse MPI of a scene is first generated using a 3D CNN, and it is further sparsely optimized by focusing only on important MPI gradients in a few iterations. Nevertheless, relying solely on optimization can lead to artifacts at occlusion boundaries. Therefore, we propose an occlusion-aware iterative refinement module that removes visual artifacts in occluded regions by iteratively filtering the input. Extensive experiments demonstrate that our method achieves comparable visual quality while being 100x faster on average than state-of-the-art offline methods and delivering better performance (about 2 dB higher in PSNR) compared to other online approaches.

翻译：随着扩展现实（XR）技术的兴起，从稀疏视角输入实时生成光场的需求日益增长。现有方法可分为离线技术和在线方法：前者虽能生成高质量的新视角图像，但推理/训练时间较长；后者则要么缺乏泛化能力，要么生成结果不理想。然而，我们观察到多平面图像（MPI）的内在稀疏流形特性，能够在保持渲染质量的同时大幅加速光场生成。基于这一认识，我们提出了EffLiFe——一种新颖的光场优化方法，它利用提出的分层稀疏梯度下降（HSGD）从稀疏视角图像实时生成高质量光场。具体技术上，首先通过3D CNN生成场景的粗略MPI，随后仅需在少量迭代中聚焦于关键MPI梯度进行稀疏优化。然而，仅依赖优化会导致遮挡边界处产生伪影。为此，我们提出了一种遮挡感知迭代细化模块，通过对输入进行迭代滤波来消除遮挡区域中的视觉伪影。大量实验表明，我们的方法在实现可比视觉质量的同时，平均比最先进的离线方法快100倍，并且相比其他在线方法获得了更优性能（PSNR提升约2 dB）。