Image-based head swapping task aims to stitch a source head to another source body flawlessly. This seldom-studied task faces two major challenges: 1) Preserving the head and body from various sources while generating a seamless transition region. 2) No paired head swapping dataset and benchmark so far. In this paper, we propose a semantic-mixing diffusion model for head swapping (HS-Diffusion) which consists of a latent diffusion model (LDM) and a semantic layout generator. We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator, achieving a coarse-grained head swapping. Semantic-mixing LDM can further implement a fine-grained head swapping with the inpainted layout as condition by a progressive fusion process, while preserving head and body with high-quality reconstruction. To this end, we propose a semantic calibration strategy for natural inpainting and a neck alignment for geometric realism. Importantly, we construct a new image-based head swapping benchmark and design two tailor-designed metrics (Mask-FID and Focal-FID). Extensive experiments demonstrate the superiority of our framework. The code will be available: https://github.com/qinghew/HS-Diffusion.

翻译：基于图像的换头任务旨在将源头部无缝拼接至另一源身体。这一鲜有研究的任务面临两大挑战：1) 在生成平滑过渡区域的同时保持不同来源头部和身体的完整性；2) 目前尚无成对的换头数据集与基准。本文提出用于换头的语义融合扩散模型（HS-Diffusion），该模型由潜在扩散模型（LDM）与语义布局生成器组成。我们融合源头部与源身体的语义布局，通过语义布局生成器对过渡区域进行修复，实现粗粒度换头；语义融合LDM则以修复后的布局为条件，通过渐进式融合过程进一步实现细粒度换头，同时以高质量重建保持头部与身体特征。为此，我们提出用于自然修复的语义校准策略与用于几何真实感的颈部对齐方法。更重要的是，我们构建了新的基于图像的换头基准，并设计了两种定制化评估指标（Mask-FID与Focal-FID）。大量实验证明了本框架的优越性。代码开源地址：https://github.com/qinghew/HS-Diffusion。