Light fields are a type of image data that capture both spatial and angular scene information by recording light rays emitted by a scene from different orientations. In this context, spatial information is defined as features that remain static regardless of perspectives, while angular information refers to features that vary between viewpoints. We propose a novel neural network that, by design, can separate angular and spatial information of a light field. The network represents spatial information using spatial kernels shared among all Sub-Aperture Images (SAIs), and angular information using sets of angular kernels for each SAI. To further improve the representation capability of the network without increasing parameter number, we also introduce angular kernel allocation and kernel tensor decomposition mechanisms. Extensive experiments demonstrate the benefits of information separation: when applied to the compression task, our network outperforms other state-of-the-art methods by a large margin. And angular information can be easily transferred to other scenes for rendering dense views, showing the successful separation and the potential use case for the view synthesis task. We plan to release the code upon acceptance of the paper to encourage further research on this topic.
翻译:光场是一种通过记录场景从不同方向发射的光线来同时捕捉空间和角度场景信息的图像数据。在此背景下,空间信息定义为不随视角变化的静态特征,而角度信息则指代随视点变化的特征。我们提出了一种新型神经网络,其设计能够分离光场的角度与空间信息。该网络通过所有子孔径图像(SAIs)共享的空间核表示空间信息,并为每个SAI使用一组角度核表示角度信息。为了在不增加参数数量的前提下进一步提升网络表征能力,我们还引入了角度核分配与核张量分解机制。大量实验证明了信息分离的优势:当应用于压缩任务时,我们的网络以较大优势优于其他最新技术。此外,角度信息可轻松迁移至其他场景用于密集视点渲染,展示了成功的分离效果及其在视图合成任务中的潜在应用。论文接收后,我们计划公开代码以推动该方向的进一步研究。