Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images. However, despite the visually impressive results, these models often struggle to preserve plausible human structure in the generations. Due to this reason, while generative models have shown promising results in aiding downstream image recognition tasks by generating large volumes of synthetic data, they remain infeasible for improving downstream human pose perception and understanding. In this work, we propose Diffusion model with Human Pose Correction (Diffusion HPC), a text-conditioned method that generates photo-realistic images with plausible posed humans by injecting prior knowledge about human body structure. We show that Diffusion HPC effectively improves the realism of human generations. Furthermore, as the generations are accompanied by 3D meshes that serve as ground truths, Diffusion HPC's generated image-mesh pairs are well-suited for downstream human mesh recovery task, where a shortage of 3D training data has long been an issue.

翻译：近期文本到图像生成模型在生成高保真、照片级真实感的图像方面展现出卓越能力。然而，尽管视觉效果令人印象深刻，这些模型在保持生成图像中人类结构的合理性方面仍常遇到困难。正因如此，尽管生成模型通过生成大量合成数据在辅助下游图像识别任务中展现出潜力，但其在改善下游人体姿态感知与理解方面仍不可行。本文提出一种结合人体姿态校正的扩散模型（Diffusion-HPC），这是一种基于文本条件的生成方法，通过注入人体结构先验知识，生成具有合理人体姿态的逼真图像。研究表明，Diffusion-HPC有效提升了人体生成的逼真度。此外，由于生成图像伴随作为真实标注的三维网格，Diffusion-HPC生成的图像-网格配对数据特别适用于长期面临三维训练数据短缺的下游人体网格重建任务。