We present a novel technique for implicit neural representation of light fields at continuously defined viewpoints with high quality and fidelity. Our implicit neural representation maps 4D coordinates defining two-plane parameterization of the light fields to the corresponding color values. We leverage periodic activations to achieve high expressivity and accurate reconstruction for complex data manifolds while keeping low storage and inference time requirements. However, na\"ively trained non-3D structured networks do not adequately satisfy the multi-view consistency; instead, they perform alpha blending of nearby viewpoints. In contrast, our View Correspondence Network, or VICON, leverages stereo matching, optimization by automatic differentiation with respect to the input space, and multi-view pixel correspondence to provide a novel implicit representation of the light fields faithful to the novel views that are unseen during the training. Experimental results show VICON superior to the state-of-the-art non-3D implicit light field representations both qualitatively and quantitatively. Moreover, our implicit representation captures a larger field of view (FoV), surpassing the extent of the observable scene by the cameras of the ground truth renderings.
翻译:我们提出了一种新技术,用于在连续定义视点下实现高保真度与高精度的隐式神经光场表示。该隐式神经表示将定义光场双平面参数化的四维坐标映射至对应颜色值。通过利用周期激活函数,我们在保持低存储与低推理时间需求的同时,实现了复杂数据流形的高表达力与精确重建。然而,朴素训练的非三维结构网络无法充分满足多视点一致性要求,反而会执行邻近视点的阿尔法混合。相比之下,我们的视点对应网络(VICON)通过结合立体匹配、基于输入空间自动微分的优化以及多视点像素对应,提供了一种忠于训练中未见新视角的光场隐式表示。实验结果表明,VICON在定性与定量上均优于现有的非三维隐式光场表示方法。此外,我们的隐式表示能够捕获更大的视场(FoV),其范围超越了真实渲染相机所观测到的场景范围。