Despite advances in dance generation, most methods are trained in the skeletal domain and ignore mesh-level physical constraints. As a result, motions that look plausible as joint trajectories often exhibit body self-penetration and Foot-Ground Contact (FGC) anomalies when visualized with a human body mesh, reducing the aesthetic appeal of generated dances and limiting their real-world applications. We address this skeleton-to-mesh gap by deriving physics-based rewards from the body mesh and applying Reinforcement Learning Fine-Tuning (RLFT) to steer the diffusion model toward physically plausible motion synthesis under mesh visualization. Our reward design combines (i) an imitation reward that measures a motion's general plausibility by its imitability in a physical simulator (penalizing penetration and foot skating), and (ii) a Foot-Ground Deviation (FGD) reward with test-time FGD guidance to better capture the dynamic foot-ground interaction in dance. However, we find that the physics-based rewards tend to push the model to generate freezing motions for fewer physical anomalies and better imitability. To mitigate it, we propose an anti-freezing reward to preserve motion dynamics while maintaining physical plausibility. Experiments on multiple dance datasets consistently demonstrate that our method can significantly improve the physical plausibility of generated motions, yielding more realistic and aesthetically pleasing dances. The project page is available at: https://jjd1123.github.io/Skeleton2Stage/
翻译:尽管舞蹈生成技术已取得进展,但多数方法仅在骨骼域进行训练,忽略了网格层级的物理约束。因此,在关节轨迹层面看似合理的动作,当使用人体网格可视化时,常出现身体自穿透和足部-地面接触异常现象,这降低了生成舞蹈的观赏性并限制了其实际应用。我们通过从人体网格推导基于物理的奖励函数,并应用强化学习微调技术引导扩散模型在网格可视化条件下生成物理合理的动作,从而弥合骨骼域与网格域之间的差距。我们的奖励设计包含:(1)模仿奖励——通过动作在物理模拟器中的可模仿性衡量其整体合理性(惩罚穿透与足部滑动现象);(2)足部-地面偏差奖励——结合测试时FGD引导机制,以更好地捕捉舞蹈中动态的足地交互。然而,我们发现基于物理的奖励倾向于驱使模型生成冻结式动作以减少物理异常并提升可模仿性。为缓解此问题,我们提出抗冻结奖励机制,在保持物理合理性的同时保留动作动态特性。在多个舞蹈数据集上的实验一致表明,本方法能显著提升生成动作的物理合理性,产生更逼真且具观赏性的舞蹈。项目页面详见:https://jjd1123.github.io/Skeleton2Stage/