Network foundation models promise reusable representations for diverse traffic analysis tasks, but recent diagnostic works have revealed fundamental problems: models exploit dataset shortcuts rather than learning genuine traffic patterns, produce collapsed embedding spaces, and fail to capture the exogenous network conditions that shape real-world behavior. We translate these diagnostic insights into four concrete design principles: protocol-aware tokenization, operational context embedding, burst-flow hierarchical attention, and privacy-by-construction input design, and build netFound, a network foundation model whose architecture is motivated by this failure analysis. We pretrain netFound on a billion-token-scale corpus over 5000 GPU hours, and demonstrate that it produces high-quality representations with lower anisotropy, significantly higher alignment with domain-expert features, and an F1 of 0.95 on exogenous context discrimination where existing state-of-the-art models score below 0.62, while preserving privacy by excluding payload and IP addresses. netFound demonstrates significant improvements in frozen-encoder evaluation, showing that pretrained embeddings themselves carry useful structure, and remains the top performer across all benchmarks in end-to-end fine-tuned settings. We release full open-source code, weights for three model sizes on HuggingFace, a containerized pipeline from raw PCAPs to downstream inference, and the full 4.2 billion flows pretraining dataset to facilitate reproducibility and further research.
翻译:网络基础模型有望为多样化的流量分析任务提供可复用的表示,但近期诊断性工作揭示了根本性问题:模型利用数据集捷径而非学习真实流量模式,产生坍塌的嵌入空间,并且无法捕捉塑造真实世界行为的外生网络条件。我们将这些诊断性见解转化为四项具体设计原则——协议感知分词化、运行环境上下文嵌入、突发流层次注意力,以及自带隐私保护的输入设计,并基于此故障分析构建了网络基础模型netFound。我们在超过5000 GPU小时、十亿级token的语料上预训练netFound,证明其能生成高质量表示:各向异性更低,与领域专家特征的对齐显著增强,在外生性上下文判别任务中F1值达0.95(现有最优模型低于0.62),同时通过排除载荷和IP地址实现隐私保护。netFound在冻结编码器评估中展现显著改进,表明预训练嵌入本身已携带有效结构,并在端到端微调设置下保持所有基准测试的最佳性能。我们发布完整开源代码、HuggingFace上三种模型规模的权重、从原始PCAP文件到下游推理的容器化流水线,以及完整的42亿条流预训练数据集,以促进可复现性和后续研究。