Cyber Threat Intelligence (CTI) reporting is pivotal in contemporary risk management strategies. As the volume of CTI reports continues to surge, the demand for automated tools to streamline report generation becomes increasingly apparent. While Natural Language Processing techniques have shown potential in handling text data, they often struggle to address the complexity of diverse data sources and their intricate interrelationships. Moreover, established paradigms like STIX have emerged as de facto standards within the CTI community, emphasizing the formal categorization of entities and relations to facilitate consistent data sharing. In this paper, we introduce AGIR (Automatic Generation of Intelligence Reports), a transformative Natural Language Generation tool specifically designed to address the pressing challenges in the realm of CTI reporting. AGIR's primary objective is to empower security analysts by automating the labor-intensive task of generating comprehensive intelligence reports from formal representations of entity graphs. AGIR utilizes a two-stage pipeline by combining the advantages of template-based approaches and the capabilities of Large Language Models such as ChatGPT. We evaluate AGIR's report generation capabilities both quantitatively and qualitatively. The generated reports accurately convey information expressed through formal language, achieving a high recall value (0.99) without introducing hallucination. Furthermore, we compare the fluency and utility of the reports with state-of-the-art approaches, showing how AGIR achieves higher scores in terms of Syntactic Log-Odds Ratio (SLOR) and through questionnaires. By using our tool, we estimate that the report writing time is reduced by more than 40%, therefore streamlining the CTI production of any organization and contributing to the automation of several CTI tasks.
翻译:网络威胁情报(CTI)报告在当代风险管理策略中占据核心地位。随着CTI报告数量的持续激增,对于自动化工具来简化报告生成流程的需求日益显著。尽管自然语言处理技术在处理文本数据方面展现出潜力,但其往往难以应对多样化数据源及其复杂内在关联的挑战。此外,如STIX等成熟范式已成为CTI社区的既定标准,强调对实体和关系的正式分类以促进一致的数据共享。本文提出AGIR(智能报告自动生成工具),这是一种专门设计用于应对CTI报告领域紧迫挑战的变革性自然语言生成工具。AGIR的主要目标是通过自动化生成基于实体图形式化表示的综合性情报报告这一劳动密集型任务,赋能安全分析师。AGIR采用两阶段流水线架构,融合了基于模板的方法与ChatGPT等大型语言模型的优势。我们通过定量与定性双重维度评估了AGIR的报告生成能力。生成的报告能准确传达形式语言表达的信息,实现了高召回率(0.99)且未出现幻觉现象。此外,我们通过句法对数几率比和问卷调查对比了报告流畅度与实用性,证明AGIR在两项指标上均优于现有最优方法。应用该工具后,我们估算报告撰写时间可缩短超过40%,从而显著简化任何组织的CTI生产流程,并为多项CTI任务的自动化做出贡献。