We introduce Gaussian-Flow, a novel point-based approach for fast dynamic scene reconstruction and real-time rendering from both multi-view and monocular videos. In contrast to the prevalent NeRF-based approaches hampered by slow training and rendering speeds, our approach harnesses recent advancements in point-based 3D Gaussian Splatting (3DGS). Specifically, a novel Dual-Domain Deformation Model (DDDM) is proposed to explicitly model attribute deformations of each Gaussian point, where the time-dependent residual of each attribute is captured by a polynomial fitting in the time domain, and a Fourier series fitting in the frequency domain. The proposed DDDM is capable of modeling complex scene deformations across long video footage, eliminating the need for training separate 3DGS for each frame or introducing an additional implicit neural field to model 3D dynamics. Moreover, the explicit deformation modeling for discretized Gaussian points ensures ultra-fast training and rendering of a 4D scene, which is comparable to the original 3DGS designed for static 3D reconstruction. Our proposed approach showcases a substantial efficiency improvement, achieving a $5\times$ faster training speed compared to the per-frame 3DGS modeling. In addition, quantitative results demonstrate that the proposed Gaussian-Flow significantly outperforms previous leading methods in novel view rendering quality. Project page: https://nju-3dv.github.io/projects/Gaussian-Flow

翻译：我们提出Gaussian-Flow，一种新颖的基于点的方法，用于从多视角和单目视频中实现快速动态场景重建与实时渲染。与当前受限于缓慢训练和渲染速度的NeRF方法不同，我们的方法利用了基于点的3D高斯泼溅（3DGS）最新进展。具体而言，本文提出一种新颖的双域形变模型（DDDM），用于显式建模每个高斯点的属性形变，其中每个属性的时间依赖性残差通过时域中的多项式拟合与频域中的傅里叶级数拟合来捕捉。所提出的DDDM能够对长视频片段中的复杂场景形变进行建模，无需为每帧分别训练单独的3DGS，也无需引入额外的隐式神经场来建模3D动态。此外，针对离散化高斯点的显式形变建模确保了四维场景的超快训练与渲染速度，其性能可与专为静态三维重建设计的原始3DGS相媲美。我们的方法展现出显著的效率提升，训练速度相比逐帧3DGS建模提升了5倍。同时，定量结果表明，所提出的Gaussian-Flow在新视角渲染质量上显著优于此前领先方法。项目主页：https://nju-3dv.github.io/projects/Gaussian-Flow