Face anonymization aims to conceal identity information while preserving non-identity attributes. Mainstream diffusion models rely on inference-time interventions such as negative guidance or energy-based optimization, which are applied post-training to suppress identity features. These interventions often introduce distribution shifts and entangle identity with non-identity attributes, degrading visual fidelity and data utility. To address this, we propose \textbf{ID\textsuperscript{2}Face}, a training-centric anonymization framework that removes the need for inference-time optimization. The rationale of our method is to learn a structured latent space where identity and non-identity information are explicitly disentangled, enabling direct and controllable anonymization at inference. To this end, we design a conditional diffusion model with an identity-masked learning scheme. An Identity-Decoupled Latent Recomposer uses an Identity Variational Autoencoder to model identity features, while non-identity attributes are extracted from same-identity pairs and aligned through bidirectional latent alignment. An Identity-Guided Latent Harmonizer then fuses these representations via soft-gating conditioned on noisy feature prediction. The model is trained with a recomposition-based reconstruction loss to enforce disentanglement. At inference, anonymization is achieved by sampling a random identity vector from the learned identity space. To further suppress identity leakage, we introduce an Orthogonal Identity Mapping strategy that enforces orthogonality between sampled and source identity vectors. Experiments demonstrate that ID\textsuperscript{2}Face outperforms existing methods in visual quality, identity suppression, and utility preservation.

翻译：人脸匿名化旨在隐藏身份信息的同时保留非身份属性。主流扩散模型依赖于推理时干预，如负向引导或基于能量的优化，这些方法在训练后应用以抑制身份特征。此类干预常导致分布偏移，并使身份与非身份属性相互纠缠，从而降低视觉保真度与数据可用性。为解决此问题，我们提出 \\textbf{ID\\textsuperscript{2}Face}，一种以训练为核心的匿名化框架，无需推理时优化。该方法的核心思想是学习一个结构化的潜在空间，其中身份与非身份信息被显式解耦，从而在推理时实现直接且可控的匿名化。为此，我们设计了一种带身份掩码学习策略的条件扩散模型。身份解耦潜在重组器利用身份变分自编码器建模身份特征，而非身份属性则从同身份配对中提取，并通过双向潜在对齐进行校准。身份引导潜在协调器随后通过基于噪声特征预测的软门控机制融合这些表征。模型通过基于重构的重组损失进行训练，以强制解耦。在推理阶段，通过从学习到的身份空间中采样随机身份向量实现匿名化。为进一步抑制身份信息泄露，我们引入正交身份映射策略，强制采样身份向量与源身份向量间的正交性。实验表明，ID\\textsuperscript{2}Face 在视觉质量、身份抑制与效用保持方面均优于现有方法。