Real-time novel-view image synthesis on mobile devices is prohibitive due to the limited computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the other hand, recent advances in neural light field representations have shown promising real-time view synthesis results on mobile devices. Neural light field methods learn a direct mapping from a ray representation to the pixel color. The current choice of ray representation is either stratified ray sampling or Plucker coordinates, overlooking the classic light slab (two-plane) representation, the preferred representation to interpolate between light field views. In this work, we find that using the light slab representation is an efficient representation for learning a neural light field. More importantly, it is a lower-dimensional ray representation enabling us to learn the 4D ray space using feature grids which are significantly faster to train and render. Although mostly designed for frontal views, we show that the light-slab representation can be further extended to non-frontal scenes using a divide-and-conquer strategy. Our method offers superior rendering quality compared to previous light field methods and achieves a significantly improved trade-off between rendering quality and speed.
翻译:在移动设备上实现实时新视角图像合成受限于有限的计算能力和存储空间。基于体渲染的方法(如NeRF及其衍生模型)因高昂的计算开销不适用于移动设备。另一方面,神经光场表示的最新进展已在移动设备上展现出颇具前景的实时视角合成效果。神经光场方法通过将光线表示直接映射到像素颜色来实现学习。当前主流的光线表示方案是分层光线采样或Plucker坐标,却忽视了经典的双平面光板(两平面)表示——这种表示本是光场视角插值的优选方案。本研究发现,光板表示是学习神经光场的有效编码方式,更重要的是,这种低维光线表示使我们能够利用特征网格学习4D光线空间,显著提升训练和渲染速度。尽管光板表示主要针对正面视角设计,我们通过分治策略将其扩展至非正面场景。与先前光场方法相比,本方法在渲染质量上表现更优,并实现了渲染质量与速度之间的显著改善的平衡。