Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images. However, despite the visually impressive results, these models often struggle to preserve plausible human structure in the generations. Due to this reason, while generative models have shown promising results in aiding downstream image recognition tasks by generating large volumes of synthetic data, they are not suitable for improving downstream human pose perception and understanding. In this work, we propose a Diffusion model with Human Pose Correction (Diffusion-HPC), a text-conditioned method that generates photo-realistic images with plausible posed humans by injecting prior knowledge about human body structure. Our generated images are accompanied by 3D meshes that serve as ground truths for improving Human Mesh Recovery tasks, where a shortage of 3D training data has long been an issue. Furthermore, we show that Diffusion-HPC effectively improves the realism of human generations under varying conditioning strategies.

翻译：近期文本到图像生成模型在生成高保真、照片级真实感图像方面展现出卓越能力。然而，尽管视觉结果令人印象深刻，这些模型在生成过程中往往难以保持合理的人体结构。正因如此，虽然生成模型通过产生大量合成数据在下游图像识别任务中展现出可喜成果，但其并不适用于提升下游人体姿态感知与理解能力。本文提出一种基于人体姿态修正的扩散模型（Diffusion-HPC），这是一种通过注入人体结构先验知识生成具有合理姿态人物的照片级真实感图像的文本条件方法。生成的图像附有三维网格，可作为改进人体网格恢复任务的真实标注——该领域长期面临三维训练数据短缺的困境。此外，我们证明Diffusion-HPC能在不同条件生成策略下有效提升人体生成的真实性。