3D Gaussian Splatting (3DGS) has recently emerged as a promising contender to Neural Radiance Fields (NeRF) in 3D scene reconstruction and real-time novel view synthesis. 3DGS outperforms NeRF in training and inference speed but has substantially higher storage requirements. To remedy this downside, we propose POTR, a post-training 3DGS codec built on two novel techniques. First, POTR introduces a novel pruning approach that uses a modified 3DGS rasterizer to efficiently calculate every splat's individual removal effect simultaneously. This technique results in 2-4x fewer splats than other post-training pruning techniques and as a result also significantly accelerates inference with experiments demonstrating 1.5-2x faster inference than other compressed models. Second, we propose a novel method to recompute lighting coefficients, significantly reducing their entropy without using any form of training. Our fast and highly parallel approach especially increases AC lighting coefficient sparsity, with experiments demonstrating increases from 70% to 97%, with minimal loss in quality. Finally, we extend POTR with a simple fine-tuning scheme to further enhance pruning, inference, and rate-distortion performance. Experiments demonstrate that POTR, even without fine-tuning, consistently outperforms all other post-training compression techniques in both rate-distortion performance and inference speed.
翻译:3D高斯泼溅(3DGS)近期已成为神经辐射场(NeRF)在三维场景重建与实时新视角合成领域的重要竞争者。3DGS在训练与推理速度上优于NeRF,但其存储需求显著更高。为弥补此缺陷,我们提出POTR——一种基于两项创新技术的训练后3DGS编解码器。首先,POTR引入了一种新颖的剪枝方法,通过改进的3DGS光栅化器同步高效计算每个泼溅体的独立移除效应。该技术使泼溅体数量比其他训练后剪枝技术减少2-4倍,从而显著加速推理,实验证明其推理速度比其他压缩模型快1.5-2倍。其次,我们提出一种重新计算光照系数的创新方法,在不依赖任何训练形式的前提下显著降低其熵值。我们快速且高度并行的方案特别提升了AC光照系数的稀疏性,实验表明稀疏度从70%增至97%,且质量损失极小。最后,我们通过简单微调方案扩展POTR,进一步提升剪枝效果、推理速度与率失真性能。实验证明,即使未经微调,POTR在率失真性能与推理速度方面均持续优于所有其他训练后压缩技术。