Natural Language Processing (NLP) domain is experiencing a revolution due to the capabilities of Pre-trained Large Language Models ( LLMs), fueled by ground-breaking Transformers architecture, resulting into unprecedented advancements. Their exceptional aptitude for assessing probability distributions of text sequences is the primary catalyst for outstanding improvement of both the precision and efficiency of NLP models. This paper introduces for the first time SecurityLLM, a pre-trained language model designed for cybersecurity threats detection. The SecurityLLM model is articulated around two key generative elements: SecurityBERT and FalconLLM. SecurityBERT operates as a cyber threat detection mechanism, while FalconLLM is an incident response and recovery system. To the best of our knowledge, SecurityBERT represents the inaugural application of BERT in cyber threat detection. Despite the unique nature of the input data and features, such as the reduced significance of syntactic structures in content classification, the suitability of BERT for this duty demonstrates unexpected potential, thanks to our pioneering study. We reveal that a simple classification model, created from scratch, and consolidated with LLMs, exceeds the performance of established traditional Machine Learning (ML) and Deep Learning (DL) methods in cyber threat detection, like Convolutional Neural Networks (CNN) or Recurrent Neural Networks (RNN). The experimental analysis, conducted using a collected cybersecurity dataset, proves that our SecurityLLM model can identify fourteen (14) different types of attacks with an overall accuracy of 98%

翻译：自然语言处理（NLP）领域正因预训练大型语言模型（LLMs）的能力而经历一场革命，这一变革得益于突破性的Transformer架构，从而带来了前所未有的进步。这些模型在评估文本序列概率分布方面的卓越能力，是NLP模型精度和效率显著提升的主要催化剂。本文首次提出SecurityLLM，这是一种专为网络安全威胁检测设计的预训练语言模型。SecurityLLM模型围绕两个关键生成元素构建：SecurityBERT和FalconLLM。SecurityBERT作为网络威胁检测机制运行，而FalconLLM则是一个事件响应与恢复系统。据我们所知，SecurityBERT代表了BERT在网络威胁检测中的首次应用。尽管输入数据和特征具有独特性，例如句法结构在内容分类中的重要性降低，但通过我们的开创性研究，BERT在此任务中的适用性展现出了意想不到的潜力。我们揭示出一个从零开始构建并与LLMs结合的简单分类模型，其性能超越了卷积神经网络（CNN）或循环神经网络（RNN）等传统机器学习（ML）和深度学习（DL）方法在网络威胁检测中的应用。使用收集的网络安全数据集进行的实验分析证明，我们的SecurityLLM模型能够识别十四（14）种不同类型的攻击，总体准确率达到98%。