Representing networks as a graph and training a link prediction model using benign connections is an effective method of anomaly-based intrusion detection. Existing works using this technique have shown great success using temporal graph neural networks and skip-gram-based approaches on random walks. However, random walk-based approaches are unable to incorporate rich edge data, while the GNN-based approaches require large amounts of memory to train. In this work, we propose extending the original insight from random walk-based skip-grams--that random walks through a graph are analogous to sentences in a corpus--to the more modern transformer-based foundation models. Using language models that take advantage of GPU optimizations, we can quickly train a graph foundation model to predict missing tokens in random walks through a network of computers. The graph foundation model is then finetuned for link prediction and used as a network anomaly detector. This new approach allows us to combine the efficiency of random walk-based methods and the rich semantic representation of deep learning methods. This system, which we call CyberGFM, achieved state-of-the-art results on three widely used network anomaly detection datasets, delivering a up to 2$\times$ improvement in average precision. We found that CyberGFM outperforms all prior works in unsupervised link prediction for network anomaly detection, using the same number of parameters, and with equal or better efficiency than the previous best approaches.

翻译：将网络表示为图，并利用良性连接训练链路预测模型，是一种基于异常的入侵检测有效方法。现有研究采用该技术，在时序图神经网络和基于随机游走的跳元模型方法上已取得显著成功。然而，基于随机游走的方法无法融入丰富的边数据，而基于图神经网络的方法则需要大量内存进行训练。本研究提出将基于随机游走的跳元模型的原始洞见——图中随机游走与语料库中的句子具有相似性——扩展至更现代的基于Transformer的基础模型。通过利用GPU优化的语言模型，我们能够快速训练一个图基础模型，以预测计算机网络中随机游走的缺失标记。随后，该图基础模型针对链路预测任务进行微调，并用作网络异常检测器。这一新方法使我们能够结合基于随机游走方法的高效性与深度学习方法的丰富语义表示能力。我们称之为CyberGFM的系统在三个广泛使用的网络异常检测数据集上取得了最先进的性能，平均精度提升高达2倍。研究发现，在参数数量相同且效率不低于先前最佳方法的前提下，CyberGFM在网络异常检测的无监督链路预测任务中超越了所有现有工作。