The task of 4D content generation involves creating dynamic 3D models that evolve over time in response to specific input conditions, such as images. Existing methods rely heavily on pre-trained video diffusion models to guide 4D content dynamics, but these approaches often fail to capture essential physical principles, as video diffusion models lack a robust understanding of real-world physics. Moreover, these models face challenges in providing fine-grained control over dynamics and exhibit high computational costs. In this work, we propose Phys4DGen, a novel, high-efficiency framework that generates physics-compliant 4D content from a single image with enhanced control capabilities. Our approach uniquely integrates physical simulations into the 4D generation pipeline, ensuring adherence to fundamental physical laws. Inspired by the human ability to infer physical properties visually, we introduce a Physical Perception Module (PPM) that discerns the material properties and structural components of the 3D object from the input image, facilitating accurate downstream simulations. Phys4DGen significantly accelerates the 4D generation process by eliminating iterative optimization steps in the dynamics modeling phase. It allows users to intuitively control the movement speed and direction of generated 4D content by adjusting external forces, achieving finely tunable, physically plausible animations. Extensive evaluations show that Phys4DGen outperforms existing methods in both inference speed and physical realism, producing high-quality, controllable 4D content. Our project page is available at the link: \url{https://jiajinglin.github.io/Phys4DGen/}.
翻译:四维内容生成任务旨在创建能够随时间演化、并响应特定输入条件(如图像)的动态三维模型。现有方法严重依赖预训练的视频扩散模型来引导四维内容的动态变化,但由于视频扩散模型缺乏对现实世界物理原理的深刻理解,这些方法往往无法捕捉关键的物理规律。此外,现有模型在实现对动态过程的细粒度控制方面存在局限,且计算成本高昂。本研究提出Phys4DGen,一种新颖的高效框架,能够从单张图像生成符合物理规律且具有增强控制能力的四维内容。我们的方法创新性地将物理仿真集成到四维生成流程中,确保生成内容遵循基本物理定律。受人类通过视觉推断物理属性能力的启发,我们引入了物理感知模块(PPM),该模块能够从输入图像中识别三维物体的材料属性与结构组成,从而为下游的精确仿真提供支持。Phys4DGen通过在动态建模阶段消除迭代优化步骤,显著加速了四维内容的生成过程。该框架允许用户通过调整外力来直观控制生成四维内容的运动速度与方向,实现可精细调节且物理合理的动画效果。大量实验评估表明,Phys4DGen在推理速度与物理真实性方面均优于现有方法,能够生成高质量、可控的四维内容。项目页面详见:\url{https://jiajinglin.github.io/Phys4DGen/}。