Photorealistic reconstruction relying on 3D Gaussian Splatting has shown promising potential in various domains. However, the current 3D Gaussian Splatting system only supports radiance field reconstruction using undistorted perspective images. In this paper, we present OmniGS, a novel omnidirectional Gaussian splatting system, to take advantage of omnidirectional images for fast radiance field reconstruction. Specifically, we conduct a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting. According to the derivatives, we then implement a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto the equirectangular screen space for omnidirectional image rendering. We realize differentiable optimization of the omnidirectional radiance field without the requirement of cube-map rectification or tangent-plane approximation. Extensive experiments conducted in egocentric and roaming scenarios demonstrate that our method achieves state-of-the-art reconstruction quality and high rendering speed using omnidirectional images. The code will be publicly available.

翻译：基于3D高斯溅射的光照真实感重建已在多个领域展现出广阔前景。然而，当前的3D高斯溅射系统仅支持使用无畸变透视图像进行辐射场重建。本文提出OmniGS——一种新型全向高斯溅射系统，以利用全向图像实现快速辐射场重建。具体而言，我们对3D高斯溅射中的球面相机模型导数进行了理论分析。基于该导数，我们实现了一种新的GPU加速全向光栅化器，可直接将3D高斯函数溅射至等距柱状投影屏幕空间以进行全向图像渲染。我们实现了全向辐射场的可微分优化，无需进行立方体贴图校正或切平面近似。在自我中心视角与自由漫游场景中开展的大量实验表明，本方法在使用全向图像时实现了最优的重建质量与高速渲染性能。代码将公开提供。