Text-guided diffusion models have revolutionized image and video generation and have also been successfully used for optimization-based 3D object synthesis. Here, we instead focus on the underexplored text-to-4D setting and synthesize dynamic, animated 3D objects using score distillation methods with an additional temporal dimension. Compared to previous work, we pursue a novel compositional generation-based approach, and combine text-to-image, text-to-video, and 3D-aware multiview diffusion models to provide feedback during 4D object optimization, thereby simultaneously enforcing temporal consistency, high-quality visual appearance and realistic geometry. Our method, called Align Your Gaussians (AYG), leverages dynamic 3D Gaussian Splatting with deformation fields as 4D representation. Crucial to AYG is a novel method to regularize the distribution of the moving 3D Gaussians and thereby stabilize the optimization and induce motion. We also propose a motion amplification mechanism as well as a new autoregressive synthesis scheme to generate and combine multiple 4D sequences for longer generation. These techniques allow us to synthesize vivid dynamic scenes, outperform previous work qualitatively and quantitatively and achieve state-of-the-art text-to-4D performance. Due to the Gaussian 4D representation, different 4D animations can be seamlessly combined, as we demonstrate. AYG opens up promising avenues for animation, simulation and digital content creation as well as synthetic data generation.

翻译：文本引导扩散模型已革新了图像和视频生成，并成功应用于基于优化的3D对象合成。本文聚焦于尚未充分探索的文本到4D生成任务，采用分数蒸馏方法并引入额外时间维度，合成动态、动画化的3D对象。与先前工作不同，我们提出了新颖的组合式生成方法，通过结合文本到图像、文本到视频及3D感知多视角扩散模型，为4D对象优化过程提供反馈，同时实现时间一致性、高质量视觉外观与逼真几何结构的协同约束。我们的方法名为"对齐你的高斯"（Align Your Gaussians, AYG），采用具有形变场的动态3D高斯泼溅作为4D表示。AYG的关键创新在于通过约束运动3D高斯分布的规则化方法，稳定优化过程并诱导运动生成。此外，我们提出运动放大机制与新型自回归合成方案，用于生成并组合多个4D序列以实现长时生成。这些技术使我们能够合成生动的动态场景，在定性与定量指标上均超越先前工作，达到文本到4D生成的最优性能。由于采用高斯4D表示，不同4D动画可无缝组合——我们对此进行了验证。AYG为动画制作、仿真模拟、数字内容创作以及合成数据生成开辟了广阔前景。