With the rise of Extended Reality (XR) technology, there is a growing need for real-time light field generation from sparse view inputs. Existing methods can be classified into offline techniques, which can generate high-quality novel views but at the cost of long inference/training time, and online methods, which either lack generalizability or produce unsatisfactory results. However, we have observed that the intrinsic sparse manifold of Multi-plane Images (MPI) enables a significant acceleration of light field generation while maintaining rendering quality. Based on this insight, we introduce EffLiFe, a novel light field optimization method, which leverages the proposed Hierarchical Sparse Gradient Descent (HSGD) to produce high-quality light fields from sparse view images in real time. Technically, the coarse MPI of a scene is first generated using a 3D CNN, and it is further sparsely optimized by focusing only on important MPI gradients in a few iterations. Nevertheless, relying solely on optimization can lead to artifacts at occlusion boundaries. Therefore, we propose an occlusion-aware iterative refinement module that removes visual artifacts in occluded regions by iteratively filtering the input. Extensive experiments demonstrate that our method achieves comparable visual quality while being 100x faster on average than state-of-the-art offline methods and delivering better performance (about 2 dB higher in PSNR) compared to other online approaches.

翻译：随着扩展现实（XR）技术的兴起，从稀疏视角输入中实时生成光场的需求日益增长。现有方法可分为离线技术和在线方法两类：前者能够生成高质量新视角图像，但推理/训练时间较长；后者则存在泛化能力不足或生成结果不理想的问题。然而，我们发现多平面图像（MPI）的固有稀疏流形特性，可以在保证渲染质量的同时显著加速光场生成。基于这一洞察，我们提出EffLiFe——一种新型光场优化方法，通过所提出的层级稀疏梯度下降（HSGD）从稀疏视角图像中实时生成高质量光场。在技术实现上，首先利用3D CNN生成场景的粗略MPI，然后通过仅关注少量迭代中的重要MPI梯度进行稀疏优化。然而，仅依赖优化会在遮挡边界产生伪影。为此，我们提出遮挡感知迭代优化模块，通过迭代过滤输入来消除遮挡区域的视觉伪影。大量实验表明，与最先进的离线方法相比，本方法在实现同等视觉质量的同时，平均速度快100倍；与其他在线方法相比，PSNR指标提升约2 dB，性能更优。