To address the increasing complexity and frequency of cybersecurity incidents emphasized by the recent cybersecurity threat reports with over 10 billion instances, cyber threat intelligence (CTI) plays a critical role in the modern cybersecurity landscape by offering the insights required to understand and combat the constantly evolving nature of cyber threats. Inspired by the powerful capability of large language models (LLMs) in handling complex tasks, in this paper, we introduce a framework to benchmark, elicit, and improve cybersecurity incident analysis and response abilities in LLMs for Security Events (SEvenLLM). Specifically, we create a high-quality bilingual instruction corpus by crawling cybersecurity raw text from cybersecurity websites to overcome the lack of effective data for information extraction. Then, we design a pipeline to auto-select tasks from the tasks pool and convert the raw text into supervised corpora comprised of question and response. The instruction dataset SEvenLLM-Instruct is used to train cybersecurity LLMs with the multi-task learning objective (27 well-designed tasks) for augmenting the analysis of cybersecurity events. Extensive experiments in our curated benchmark (SEvenLLM-bench) demonstrate that SEvenLLM performs more sophisticated threat analysis and fortifies defenses against the evolving landscape of cyber threats.

翻译：为应对近期网络安全威胁报告中强调的、涉及超过100亿个实例的日益复杂和频繁的网络安全事件，网络威胁情报（CTI）通过提供理解和对抗不断演变的网络威胁所需的洞察力，在现代网络安全格局中发挥着关键作用。受大语言模型（LLM）处理复杂任务强大能力的启发，本文提出一个框架，用于对安全事件中大语言模型（SEvenLLM）进行网络安全事件分析与响应能力的基准测试、激发与改进。具体而言，我们通过爬取网络安全网站的原始文本构建了一个高质量双语指令语料库，以克服信息提取有效数据匮乏的问题。随后，我们设计了一条流水线，从任务池中自动选择任务，并将原始文本转换为由问题和回答组成的监督语料。利用指令数据集SEvenLLM-Instruct，结合多任务学习目标（27个精心设计的任务）训练网络安全大语言模型，以增强网络安全事件分析能力。在我们策划的基准测试（SEvenLLM-bench）上进行的大量实验表明，SEvenLLM能够执行更复杂的威胁分析，并强化针对不断演变的网络威胁的防御能力。