3D Gaussian Splatting (3DGS) has shown remarkable success in synthesizing novel views given multiple views of a static scene. Yet, 3DGS faces challenges when applied to dynamic scenes because 3D Gaussian parameters need to be updated per timestep, requiring a large amount of memory and at least a dozen observations per timestep. To address these limitations, we present a compact dynamic 3D Gaussian representation that models positions and rotations as functions of time with a few parameter approximations while keeping other properties of 3DGS including scale, color and opacity invariant. Our method can dramatically reduce memory usage and relax a strict multi-view assumption. In our experiments on monocular and multi-view scenarios, we show that our method not only matches state-of-the-art methods, often linked with slower rendering speeds, in terms of high rendering quality but also significantly surpasses them by achieving a rendering speed of $118$ frames per second (FPS) at a resolution of 1,352$\times$1,014 on a single GPU.
翻译:三维高斯溅射(3DGS)在给定静态场景多视角图像进行新视图合成方面已展现出显著成功。然而,当应用于动态场景时,3DGS面临挑战,因为三维高斯参数需要随时间步更新,这需要大量内存且每个时间步至少需要十余次观测。为克服这些限制,我们提出一种紧凑动态三维高斯表示方法,该方法将位置和旋转建模为时间的函数,并通过少量参数近似实现,同时保持3DGS的其他属性(包括尺度、颜色和不透明度)不变。我们的方法能显著降低内存使用量并放宽严格的多视角假设。在单目与多视角场景的实验中,我们证明该方法不仅在高渲染质量方面与常伴随较慢渲染速度的先进方法相当,更以单GPU在1,352×1,014分辨率下实现每秒$118$帧(FPS)的渲染速度而显著超越现有方法。