Network traffic refers to the amount of data being sent and received over the internet or any system that connects computers. Analyzing and understanding network traffic is vital for improving network security and management. However, the analysis of network traffic is challenging due to the diverse nature of data packets, which often feature heterogeneous headers and encrypted payloads lacking semantics. To capture the latent semantics of traffic, a few studies have adopted pre-training techniques based on the Transformer encoder or decoder to learn the representations from massive traffic data. However, these methods typically excel in traffic understanding (classification) or traffic generation tasks. To address this issue, we develop Lens, a foundation model for network traffic that leverages the T5 architecture to learn the pre-trained representations from large-scale unlabeled data. Harnessing the strength of the encoder-decoder framework, which captures the global information while preserving the generative ability, our model can better learn the representations from raw data. To further enhance pre-training effectiveness, we design a novel loss that combines three distinct tasks: Masked Span Prediction (MSP), Packet Order Prediction (POP), and Homologous Traffic Prediction (HTP). Evaluation results across various benchmark datasets demonstrate that the proposed Lens outperforms the baselines in most downstream tasks related to both traffic understanding and generation. Notably, it also requires much less labeled data for fine-tuning compared to current methods.

翻译：网络流量是指通过互联网或任何连接计算机的系统发送和接收的数据量。分析和理解网络流量对于提升网络安全和管理至关重要。然而，由于数据包的多样性（通常包含异构头部和缺乏语义的加密载荷），网络流量的分析极具挑战性。为捕捉流量的潜在语义，少数研究采用了基于Transformer编码器或解码器的预训练技术，从海量流量数据中学习表示。然而，这些方法通常仅在流量理解（分类）或流量生成任务中表现优异。为解决这一问题，我们开发了Lens——一个基于T5架构的网络流量基础模型，可从大规模无标签数据中学习预训练表示。利用编码器-解码器框架的独特优势（既能捕获全局信息又能保持生成能力），我们的模型能更好地从原始数据中学习表示。为进一步提升预训练效果，我们设计了一种融合三种不同任务的新型损失函数：掩码跨度预测（MSP）、数据包顺序预测（POP）和同源流量预测（HTP）。在多个基准数据集上的评估结果表明，所提出的Lens在绝大多数与流量理解和生成相关的下游任务中优于基线方法。值得注意的是，与现有方法相比，它在微调过程中所需的标注数据量也显著减少。