Neural rendering techniques, including NeRF and Gaussian Splatting (GS), rely on photometric consistency to produce high-quality reconstructions. However, in real-world scenarios, it is challenging to guarantee perfect photometric consistency in acquired images. Appearance codes have been widely used to address this issue, but their modeling capability is limited, as a single code is applied to the entire image. Recently, the bilateral grid was introduced to perform pixel-wise color mapping, but it is difficult to optimize and constrain effectively. In this paper, we propose a novel multi-scale bilateral grid that unifies appearance codes and bilateral grids. We demonstrate that this approach significantly improves geometric accuracy in dynamic, decoupled autonomous driving scene reconstruction, outperforming both appearance codes and bilateral grids. This is crucial for autonomous driving, where accurate geometry is important for obstacle avoidance and control. Our method shows strong results across four datasets: Waymo, NuScenes, Argoverse, and PandaSet. We further demonstrate that the improvement in geometry is driven by the multi-scale bilateral grid, which effectively reduces floaters caused by photometric inconsistency.
翻译:神经渲染技术,包括NeRF和高斯泼溅(GS),依赖于光度一致性来生成高质量重建。然而,在现实场景中,保证采集图像具有完美的光度一致性具有挑战性。外观编码已被广泛用于解决此问题,但其建模能力有限,因为单个编码应用于整幅图像。最近,双边网格被引入以执行逐像素颜色映射,但其难以有效优化和约束。本文提出了一种新颖的多尺度双边网格,统一了外观编码与双边网格。我们证明,该方法在动态、解耦的自动驾驶场景重建中显著提高了几何精度,超越了外观编码和双边网格。这对于自动驾驶至关重要,因为精确的几何对于障碍物避让和控制非常重要。我们的方法在四个数据集上均显示出优异结果:Waymo、NuScenes、Argoverse和PandaSet。我们进一步证明,几何精度的提升由多尺度双边网格驱动,其有效减少了由光度不一致引起的漂浮物。