Mixup is a well-known data-dependent augmentation technique for DNNs, consisting of two sub-tasks: mixup generation and classification. However, the recent dominant online training method confines mixup to supervised learning (SL), and the objective of the generation sub-task is limited to selected sample pairs instead of the whole data manifold, which might cause trivial solutions. To overcome such limitations, we comprehensively study the objective of mixup generation and propose \textbf{S}cenario-\textbf{A}gnostic \textbf{Mix}up (SAMix) for both SL and Self-supervised Learning (SSL) scenarios. Specifically, we hypothesize and verify the objective function of mixup generation as optimizing local smoothness between two mixed classes subject to global discrimination from other classes. Accordingly, we propose $\eta$-balanced mixup loss for complementary learning of the two sub-objectives. Meanwhile, a label-free generation sub-network is designed, which effectively provides non-trivial mixup samples and improves transferable abilities. Moreover, to reduce the computational cost of online training, we further introduce a pre-trained version, SAMix$^\mathcal{P}$, achieving more favorable efficiency and generalizability. Extensive experiments on nine SL and SSL benchmarks demonstrate the consistent superiority and versatility of SAMix compared with existing methods.

翻译：混合增强是一种众所周知的依赖于数据的深度神经网络增强技术，包含两个子任务：混合生成和分类。然而，当前主流的在线训练方法将混合增强局限于监督学习场景，且生成子任务的目标仅限于选定的样本对而非整个数据流形，可能导致平凡解。为突破这些局限，我们全面研究了混合增强生成的目标函数，并提出了**场景无关混合增强**（SAMix），适用于监督学习和自监督学习两种场景。具体而言，我们假设并验证了混合增强生成的目标函数为：在优化两个混合类间局部平滑性的同时，兼顾与其他类别的全局判别性。据此，我们提出$\eta$平衡混合损失函数以实现两个子目标的互补学习。同时，设计了无标签生成子网络，该网络能有效提供非平凡混合样本并提升迁移能力。此外，为降低在线训练的计算成本，我们进一步引入预训练版本SAMix$^\mathcal{P}$，在效率和泛化性上表现更优。在九个监督学习和自监督学习基准上的大量实验表明，与现有方法相比，SAMix具有一致优越性和通用性。