Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders. In this paper, we systematically investigate several design choices and find three insights which simplify and improve RAE. First, we study a generalized formulation where the representation is defined as sum of the last k encoder layers rather than solely the final layer. This simple change greatly improves reconstruction without encoder finetuning or specialized data (e.g., text, faces). Second, we study the prevalent assumption that RAE (using pretrained representation as encoder) replaces representation alignment (REPA), which distills the same representation to intermediate layers instead. Through large-scale empirical analysis, we uncover a surprising finding: RAE and REPA exhibit complementary working mechanisms, allowing the same representation to be used as both encoder and target for intermediate diffusion layers. Finally, the original RAE struggles with classifier-free guidance (CFG) and requires training a second, weaker diffusion model for AutoGuidance (AG). We show that REPA itself can be viewed as x-prediction in RAE latent space. By simply re-parameterizing the output of the DiT model, it can provide guidance for "free". Overall, RAEv2 leads to more than 10x faster convergence over the original RAE, achieving a state-of-the-art gFID of 1.06 in just 80 epochs on ImageNet-256. On FDr^k, RAEv2 achieves a state-of-the-art 2.17 at just 80 epochs compared to the previous best 3.26 (800 epochs) without any post-training. This motivates EP_FID@k (epochs to reach unguided gFID <= k) as a measure of training efficiency. RAEv2 attains an EP_FID@2 of 35 epochs, versus 177 for the original RAE. We also validate our approach across diverse settings for text-to-image generation and navigation world models, showing consistent improvements. Code is available at https://raev2.github.io.

翻译：表示自编码器（RAE）采用预训练视觉编码器替代传统变分自编码器。本文系统研究了若干设计选择，提炼出三个简化并改进RAE的洞见。首先，我们探究了广义公式，将表示定义为最后k个编码器层之和（而非仅最后一层）。这一简单改动无需微调编码器或依赖特殊数据（如文本、人脸），即可显著提升重建质量。其次，我们研究了普遍假设：使用预训练表示作为编码器的RAE替代了表示对齐（REPA）——后者将相同表示蒸馏至中间层。通过大规模实证分析，我们揭示了一个惊人发现：RAE与REPA展现出互补的工作机制，使得同一表示可同时用作编码器和中间扩散层的目标。最后，原始RAE在无分类器引导（CFG）中表现不佳，需训练一个更弱的第二扩散模型用于自动引导（AG）。我们证明REPA本身可视为RAE潜在空间中的x预测。仅需对DiT模型输出进行重参数化，即可“免费”提供引导。综合而言，RAEv2相较于原始RAE实现超过10倍收敛速度提升，在ImageNet-256上仅需80个epochs即可达到1.06的最优gFID。在FDr^k指标上，RAEv2在80个epochs时达2.17最优值，而此前最优结果（3.26）需800个epochs且无需后训练。这促使我们将EP_FID@k（达到无引导gFID≤k所需的epochs）作为训练效率度量标准。RAEv2的EP_FID@2为35个epochs，而原始RAE需177个epochs。我们还在文本生成图像与导航世界模型的多样化场景中验证了方法有效性，表现出持续改进。代码已开源：https://raev2.github.io。