Novel view synthesis of dynamic scenes has been an intriguing yet challenging problem. Despite recent advancements, simultaneously achieving high-resolution photorealistic results, real-time rendering, and compact storage remains a formidable task. To address these challenges, we propose Spacetime Gaussian Feature Splatting as a novel dynamic scene representation, composed of three pivotal components. First, we formulate expressive Spacetime Gaussians by enhancing 3D Gaussians with temporal opacity and parametric motion/rotation. This enables Spacetime Gaussians to capture static, dynamic, as well as transient content within a scene. Second, we introduce splatted feature rendering, which replaces spherical harmonics with neural features. These features facilitate the modeling of view- and time-dependent appearance while maintaining small size. Third, we leverage the guidance of training error and coarse depth to sample new Gaussians in areas that are challenging to converge with existing pipelines. Experiments on several established real-world datasets demonstrate that our method achieves state-of-the-art rendering quality and speed, while retaining compact storage. At 8K resolution, our lite-version model can render at 60 FPS on an Nvidia RTX 4090 GPU.

翻译：动态场景的新视角合成一直是一个引人入胜但极具挑战性的问题。尽管近年来取得了进展，但同时实现高分辨率逼真效果、实时渲染和紧凑存储仍然是一项艰巨任务。为解决这些挑战，我们提出时空高斯特征点云渲染作为一种新颖的动态场景表示方法，由三个关键组件构成。首先，我们通过为3D高斯添加时间不透明度与参数化运动/旋转，构建了表达力强的时空高斯体。这使得时空高斯能够捕捉场景中的静态、动态及瞬态内容。其次，我们引入点云特征渲染，以神经特征替代球谐函数。这些特征在保持小规模的同时，有助于建模视角与时间依赖的外观变化。第三，我们利用训练误差与粗深度信息的引导，在现有流程难以收敛的区域采样新高斯体。在多个权威真实世界数据集上的实验表明，我们的方法在实现紧凑存储的同时，达到了最先进的渲染质量与速度。在8K分辨率下，精简版模型可在Nvidia RTX 4090 GPU上以60 FPS的帧率进行渲染。