The rapid growth of 3D Gaussian Splatting (3DGS) has revolutionized neural rendering, enabling real-time production of high-quality renderings. However, the previous 3DGS-based methods have limitations in urban scenes due to reliance on initial Structure-from-Motion(SfM) points and difficulties in rendering distant, sky and low-texture areas. To overcome these challenges, we propose a hybrid optimization method named HO-Gaussian, which combines a grid-based volume with the 3DGS pipeline. HO-Gaussian eliminates the dependency on SfM point initialization, allowing for rendering of urban scenes, and incorporates the Point Densitification to enhance rendering quality in problematic regions during training. Furthermore, we introduce Gaussian Direction Encoding as an alternative for spherical harmonics in the rendering pipeline, which enables view-dependent color representation. To account for multi-camera systems, we introduce neural warping to enhance object consistency across different cameras. Experimental results on widely used autonomous driving datasets demonstrate that HO-Gaussian achieves photo-realistic rendering in real-time on multi-camera urban datasets.

翻译：三维高斯泼溅（3DGS）的快速发展革新了神经渲染领域，实现了高质量渲染效果的实时生成。然而，现有基于3DGS的方法在城市场景中存在局限性，原因在于其依赖初始运动恢复结构（SfM）点，且难以对远景、天空及低纹理区域进行渲染。为解决上述挑战，我们提出一种名为HO-Gaussian的混合优化方法，该方法将基于网格的体素与3DGS管线相结合。HO-Gaussian消除了对SfM点初始化的依赖性，能够对城市场景进行渲染，并通过引入点稠密化（Point Densitification）机制在训练过程中提升问题区域的渲染质量。此外，我们提出用高斯方向编码（Gaussian Direction Encoding）替代渲染管线中的球谐函数，从而支持视角相关的颜色表示。为适配多相机系统，我们引入神经变形（neural warping）技术以增强不同相机间的物体一致性。在广泛使用的自动驾驶数据集上的实验结果表明，HO-Gaussian能够在多相机城市数据集上实现照片级真实感的实时渲染。