Recently, high-fidelity scene reconstruction with an optimized 3D Gaussian splat representation has been introduced for novel view synthesis from sparse image sets. Making such representations suitable for applications like network streaming and rendering on low-power devices requires significantly reduced memory consumption as well as improved rendering efficiency. We propose a compressed 3D Gaussian splat representation that utilizes sensitivity-aware vector clustering with quantization-aware training to compress directional colors and Gaussian parameters. The learned codebooks have low bitrates and achieve a compression rate of up to $31\times$ on real-world scenes with only minimal degradation of visual quality. We demonstrate that the compressed splat representation can be efficiently rendered with hardware rasterization on lightweight GPUs at up to $4\times$ higher framerates than reported via an optimized GPU compute pipeline. Extensive experiments across multiple datasets demonstrate the robustness and rendering speed of the proposed approach.

翻译：近年来，基于优化三维高斯泼溅表示的高保真场景重建已被用于从稀疏图像集合成新视角。为使此类表示适用于网络流媒体及低功耗设备渲染等场景，需显著降低内存消耗并提升渲染效率。我们提出一种压缩三维高斯泼溅表示方法，利用灵敏度感知向量聚类与量化感知训练对方向色值和高斯参数进行压缩。学习获得的码本具有低位率，在真实场景中可实现高达$31\times$的压缩率，且仅造成极小的视觉质量损失。我们证明，该压缩泼溅表示可通过轻量级GPU上的硬件光栅化高效渲染，其帧率比优化后的GPU计算管线报告值高出$4\times$。多数据集上的广泛实验验证了所提方法的鲁棒性与渲染速度。