Masked Autoencoder (MAE) has demonstrated superior performance on various vision tasks via randomly masking image patches and reconstruction. However, effective data augmentation strategies for MAE still remain open questions, different from those in contrastive learning that serve as the most important part. This paper studies the prevailing mixing augmentation for MAE. We first demonstrate that naive mixing will in contrast degenerate model performance due to the increase of mutual information (MI). To address, we propose homologous recognition, an auxiliary pretext task, not only to alleviate the MI increasement by explicitly requiring each patch to recognize homologous patches, but also to perform object-aware self-supervised pre-training for better downstream dense perception performance. With extensive experiments, we demonstrate that our proposed Mixed Autoencoder (MixedAE) achieves the state-of-the-art transfer results among masked image modeling (MIM) augmentations on different downstream tasks with significant efficiency. Specifically, our MixedAE outperforms MAE by +0.3% accuracy, +1.7 mIoU and +0.9 AP on ImageNet-1K, ADE20K and COCO respectively with a standard ViT-Base. Moreover, MixedAE surpasses iBOT, a strong MIM method combined with instance discrimination, while accelerating training by 2x. To our best knowledge, this is the very first work to consider mixing for MIM from the perspective of pretext task design. Code will be made available.

翻译：掩码自编码器（MAE）通过随机掩码图像块并进行重建，已在各类视觉任务中展现出卓越性能。然而，与作为对比学习中核心环节的有效数据增强策略不同，MAE中高效的数据增强方法仍是开放性问题。本文研究MAE中主流的混合增强技术。我们首先证明，由于互信息（MI）的提升，直接采用朴素混合增强反而会降低模型性能。为解决该问题，我们提出同源识别——一种辅助预文本任务，不仅通过显式要求每个图像块识别同源块来缓解MI增长，还能实现面向对象感知的自监督预训练，以提升下游密集感知性能。通过大量实验，我们证明提出的混合自编码器（MixedAE）在不同下游任务中，作为掩码图像建模（MIM）的增强方法，取得了最先进的迁移性能，并显著提升效率。具体而言，与标准ViT-Base相比，MixedAE在ImageNet-1K、ADE20K和COCO上分别超越MAE，达到+0.3%准确率、+1.7 mIoU和+0.9 AP。此外，MixedAE在将训练速度提升2倍的同时，超越了结合实例区分的强MIM方法iBOT。据我们所知，这是首项从预文本任务设计角度考虑MIM混合增强的工作。代码将公开提供。