Diffusion models have achieved great progress in face generation. However, these models amplify the bias in the generation process, leading to an imbalance in distribution of sensitive attributes such as age, gender and race. This paper proposes a novel solution to this problem by balancing the facial attributes of the generated images. We mitigate the bias by localizing the means of the facial attributes in the latent space of the diffusion model using Gaussian mixture models (GMM). Our motivation for choosing GMMs over other clustering frameworks comes from the flexible latent structure of diffusion model. Since each sampling step in diffusion models follows a Gaussian distribution, we show that fitting a GMM model helps us to localize the subspace responsible for generating a specific attribute. Furthermore, our method does not require retraining, we instead localize the subspace on-the-fly and mitigate the bias for generating a fair dataset. We evaluate our approach on multiple face attribute datasets to demonstrate the effectiveness of our approach. Our results demonstrate that our approach leads to a more fair data generation in terms of representational fairness while preserving the quality of generated samples.

翻译：扩散模型在人脸生成领域取得了显著进展。然而，这些模型在生成过程中放大了偏差，导致年龄、性别和种族等敏感属性在分布上失衡。本文提出了一种新颖的解决方案，通过平衡生成图像的面部属性来应对这一问题。我们利用高斯混合模型在扩散模型的潜在空间中定位面部属性的均值，从而缓解偏差。选择高斯混合模型而非其他聚类框架的动机源于扩散模型灵活的潜在结构。由于扩散模型中每个采样步骤均遵循高斯分布，我们表明拟合高斯混合模型有助于定位负责生成特定属性的子空间。此外，我们的方法无需重新训练，而是即时定位子空间并缓解偏差，以生成公平的数据集。我们在多个面部属性数据集上评估了该方法，以证明其有效性。结果表明，我们的方法在保持生成样本质量的同时，在表征公平性方面实现了更公平的数据生成。