Recently, 3D Gaussian Splatting, as a novel 3D representation, has garnered attention for its fast rendering speed and high rendering quality. However, this comes with high memory consumption, e.g., a well-trained Gaussian field may utilize three million Gaussian primitives and over 700 MB of memory. We credit this high memory footprint to the lack of consideration for the relationship between primitives. In this paper, we propose a memory-efficient Gaussian field named SUNDAE with spectral pruning and neural compensation. On one hand, we construct a graph on the set of Gaussian primitives to model their relationship and design a spectral down-sampling module to prune out primitives while preserving desired signals. On the other hand, to compensate for the quality loss of pruning Gaussians, we exploit a lightweight neural network head to mix splatted features, which effectively compensates for quality losses while capturing the relationship between primitives in its weights. We demonstrate the performance of SUNDAE with extensive results. For example, SUNDAE can achieve 26.80 PSNR at 145 FPS using 104 MB memory while the vanilla Gaussian splatting algorithm achieves 25.60 PSNR at 160 FPS using 523 MB memory, on the Mip-NeRF360 dataset. Codes are publicly available at https://runyiyang.github.io/projects/SUNDAE/.

翻译：近年来，作为一种新颖的三维表示方法，三维高斯泼溅技术因其快速渲染速度和高渲染质量而备受关注。然而，这伴随着较高的内存消耗，例如一个训练良好的高斯场可能会使用三百万个高斯原语和超过700MB内存。我们认为这种高内存占用是由于忽视了原语之间的关系所致。在本文中，我们提出了一种名为SUNDAE的内存高效高斯场，结合了谱剪枝与神经补偿。一方面，我们在高斯原语集合上构建图以建模其关系，并设计谱下采样模块在保留期望信号的同时剪除原语。另一方面，为补偿剪枝高斯原语造成的质量损失，我们利用轻量级神经网络头部混合泼溅特征，有效补偿质量损失，同时在其权重中捕捉原语间关系。我们通过大量实验结果展示了SUNDAE的性能。例如，在Mip-NeRF360数据集上，SUNDAE在使用104MB内存时，可在145FPS下达到26.80 PSNR，而原始高斯泼溅算法在使用523MB内存时，在160FPS下达到25.60 PSNR。代码已在 https://runyiyang.github.io/projects/SUNDAE/ 公开。