Recently, 3D Gaussian splatting (3D-GS) has gained popularity in novel-view scene synthesis. It addresses the challenges of lengthy training times and slow rendering speeds associated with Neural Radiance Fields (NeRFs). Through rapid, differentiable rasterization of 3D Gaussians, 3D-GS achieves real-time rendering and accelerated training. They, however, demand substantial memory resources for both training and storage, as they require millions of Gaussians in their point cloud representation for each scene. We present a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements and a coarse-to-fine training strategy for a faster and more stable optimization of the Gaussian point clouds. Our approach develops a pruning stage which results in scene representations with fewer Gaussians, leading to faster training times and rendering speeds for real-time rendering of high resolution scenes. We reduce storage memory by more than an order of magnitude all while preserving the reconstruction quality. We validate the effectiveness of our approach on a variety of datasets and scenes preserving the visual quality while consuming 10-20x lesser memory and faster training/inference speed. Project page and code is available https://efficientgaussian.github.io

翻译：摘要：近年来，三维高斯泼溅（3D-GS）在新视角场景合成中广受欢迎。该方法通过三维高斯函数的快速可微分光栅化，解决了神经辐射场（NeRF）训练耗时长、渲染速度慢的问题，实现了实时渲染与加速训练。然而，由于每个场景需在点云表示中包含数百万个高斯体，3D-GS在训练与存储过程中均需大量内存资源。本文提出一种利用量化嵌入显著降低每点内存存储需求的技术，并结合从粗到精的训练策略，实现高斯点云更快、更稳定的优化。我们设计的剪枝阶段可生成含更少高斯体的场景表示，从而加速高分辨率场景实时渲染的训练与渲染速度。在保持重建质量的前提下，我们将存储内存降低超过一个数量级。我们在多种数据集与场景中验证了方法的有效性：在保持视觉质量的同时，内存消耗减少10-20倍，训练/推理速度更快。项目页面与代码已开源：https://efficientgaussian.github.io