Network traffic classification using self-supervised pre-training models based on Masked Autoencoders (MAE) has demonstrated a huge potential. However, existing methods are confined to isolated byte-level reconstruction of individual flows, lacking adequate perception of the multi-granularity contextual relationship in traffic. To address this limitation, we propose Mean MAE (MMAE), a teacher-student MAE paradigm with flow mixing strategy for building encrypted traffic pre-training model. MMAE employs a self-distillation mechanism for teacher-student interaction, where the teacher provides unmasked flow-level semantic supervision to advance the student from local byte reconstruction to multi-granularity comprehension. To break the information bottleneck in individual flows, we introduce a dynamic Flow Mixing (FlowMix) strategy to replace traditional random masking mechanism. By constructing challenging cross-flow mixed samples with interferences, it compels the model to learn discriminative representations from distorted tokens. Furthermore, we design a Packet-importance aware Mask Predictor (PMP) equipped with an attention bias mechanism that leverages packet-level side-channel statistics to dynamically mask tokens with high semantic density. Numerous experiments on a number of datasets covering encrypted applications, malware, and attack traffic demonstrate that MMAE achieves state-of-the-art performance. The code is available at https://github.com/lx6c78/MMAE

翻译：基于掩码自编码器（Masked Autoencoder, MAE）的自监督预训练模型在网络流量分类中展现出巨大潜力。然而，现有方法局限于对单个流量进行孤立的字节级重建，缺乏对流量中多粒度上下文关系的充分感知。为解决这一局限，我们提出均值MAE（Mean MAE, MMAE），一种采用师生MAE范式并融合流混合策略的加密流量预训练模型。MMAE通过自蒸馏机制实现师生交互，其中教师提供未掩码的流级语义监督，推动学生从局部字节重建进阶至多粒度理解。为打破单条流的信息瓶颈，我们引入动态流混合（Flow Mixing, FlowMix）策略替代传统随机掩码机制。通过构建携带干扰的跨流交叉混合样本，该策略迫使模型从扭曲令牌中学习判别性表征。此外，我们设计了基于令牌重要性感知的包级掩码预测器（Packet-importance aware Mask Predictor, PMP），其配备注意力偏置机制，利用包级侧信道统计量动态掩码高语义密度的令牌。涵盖加密应用、恶意软件及攻击流量等多个数据集的大量实验表明，MMAE取得了最先进的性能。代码已开源至https://github.com/lx6c78/MMAE。