In the dynamic landscape of large enterprise cybersecurity, accurately and efficiently correlating billions of security alerts into comprehensive incidents is a substantial challenge. Traditional correlation techniques often struggle with maintenance, scaling, and adapting to emerging threats and novel sources of telemetry. We introduce GraphWeaver, an industry-scale framework that shifts the traditional incident correlation process to a data-optimized, geo-distributed graph based approach. GraphWeaver introduces a suite of innovations tailored to handle the complexities of correlating billions of shared evidence alerts across hundreds of thousands of enterprises. Key among these innovations are a geo-distributed database and PySpark analytics engine for large-scale data processing, a minimum spanning tree algorithm to optimize correlation storage, integration of security domain knowledge and threat intelligence, and a human-in-the-loop feedback system to continuously refine key correlation processes and parameters. GraphWeaver is integrated into the Microsoft Defender XDR product and deployed worldwide, handling billions of correlations with a 99% accuracy rate, as confirmed by customer feedback and extensive investigations by security experts. This integration has not only maintained high correlation accuracy but reduces traditional correlation storage requirements by 7.4x. We provide an in-depth overview of the key design and operational features of GraphWeaver, setting a precedent as the first cybersecurity company to openly discuss these critical capabilities at this level of depth.

翻译：在大型企业网络安全的动态环境中，如何准确高效地将数十亿安全告警关联为综合性事件是一项重大挑战。传统关联技术通常在维护性、可扩展性以及对新兴威胁和新型遥测源的适应性方面存在局限。本文介绍GraphWeaver——一个工业级框架，它将传统事件关联流程转变为数据优化、地理分布式图计算的方法。该框架针对数十万企业间数十亿共享证据告警的复杂关联场景，提出了一系列创新技术：包括用于大规模数据处理的地理分布式数据库与PySpark分析引擎、优化关联存储的最小生成树算法、安全领域知识与威胁情报的集成，以及持续优化关键关联流程与参数的人机协同反馈系统。GraphWeaver已集成至Microsoft Defender XDR产品并在全球部署，经客户反馈和安全专家深入调查证实，其处理数十亿关联的准确率达到99%。该集成不仅保持了高关联准确率，还将传统关联存储需求降低了7.4倍。本文深入剖析GraphWeaver的核心设计与运行特性，开创了网络安全企业在此深度层面公开探讨关键能力的先河。