Representation Autoencoders (RAEs) have improved diffusion and flow models by semantically richer latent space owing to the strongly label-wise clustered DINO features in the pretrained encoders. Yet in the distillation stage, the severe anisotropy and large curvatures caused by the rich semantic representations would hinder the convergence and performance, making the trajectory-based distillation unstable. In this work, we argue that the RAE latent space is compatible with distillation via the newly proposed Drifting Models. We first quantitatively study the curvatures and isotropy statistics across different autoencoders, and theoretically reveal that Drifting Model itself is highly likely to fail on extremely scattered spaces like reconstruction-based VAEs. These motivate us to apply the drifting paradigm directly to representation autoencoders. Our proposed method, Drift-RAE, distills pretrained flow models in RAE latent spaces using Drifting, together with insightful modifications that improve training stability by thereotically aligning drifting fields with other frameworks. Regarding the experimental evidences, we achieve 1.77 FID on ImageNet 256 dataset using only 10k distillation steps, surpassing state-of-the-art RAE distillation methods and appearing comparative with the original Drifting Model without requiring an auxiliary MAE feature extractor. The code will be made publicly available.

翻译：表示自编码器（RAEs）凭借预训练编码器中强标签聚类特性的DINO特征，在语义更丰富的潜空间中改进了扩散模型与流模型。然而在蒸馏阶段，丰富语义表示导致的严重各向异性和大曲率会阻碍收敛与性能，使基于轨迹的蒸馏变得不稳定。本文提出，通过新引入的漂移模型，RAE潜空间能够兼容蒸馏。我们首先定量分析了不同自编码器的曲率与各向同性统计量，并从理论上揭示漂移模型本身极可能在极度分散的空间（如基于重建的VAE）中失效。这些发现促使我们直接将漂移范式应用于表示自编码器。所提出的Drift-RAE方法，利用漂移技术在RAE潜空间中蒸馏预训练流模型，并辅以通过理论对齐漂移场与其他框架来提升训练稳定性的深刻改进。实验表明，我们在ImageNet 256数据集上仅用1万步蒸馏步骤即达到1.77 FID，超越了最先进的RAE蒸馏方法，且在不依赖辅助MAE特征提取器的情况下与原始漂移模型性能相当。代码将公开发布。