Matrix/array analysis of networks can provide significant insight into their behavior and aid in their operation and protection. Prior work has demonstrated the analytic, performance, and compression capabilities of GraphBLAS (graphblas.org) hypersparse matrices and D4M (d4m.mit.edu) associative arrays (a mathematical superset of matrices). Obtaining the benefits of these capabilities requires integrating them into operational systems, which comes with its own unique challenges. This paper describes two examples of real-time operational implementations. First, is an operational GraphBLAS implementation that constructs anonymized hypersparse matrices on a high-bandwidth network tap. Second, is an operational D4M implementation that analyzes daily cloud gateway logs. The architectures of these implementations are presented. Detailed measurements of the resources and the performance are collected and analyzed. The implementations are capable of meeting their operational requirements using modest computational resources (a couple of processing cores). GraphBLAS is well-suited for low-level analysis of high-bandwidth connections with relatively structured network data. D4M is well-suited for higher-level analysis of more unstructured data. This work demonstrates that these technologies can be implemented in operational settings.
翻译:采用矩阵/数组方法对网络进行深入分析,可揭示其行为规律,并辅助网络运行与安全防护。已有研究已充分论证GraphBLAS(graphblas.org)超稀疏矩阵与D4M(d4m.mit.edu)关联数组(矩阵的数学超集)在分析、性能及压缩方面的能力。然而,要将这些能力融入实际运行系统面临独特挑战。本文描述两种实时运行实现的案例:其一,基于GraphBLAS的运行实现,可在高带宽网络分接点上构建匿名化超稀疏矩阵;其二,基于D4M的运行实现,可分析每日云网关日志。本文给出了这两种实现的架构设计,并收集分析了其资源与性能的详细测量数据。结果表明,该实现仅需适度计算资源(数个处理核心)即可满足运行需求。GraphBLAS适用于对结构化程度较高的网络数据进行低层级高带宽连接分析,而D4M则适用于对非结构化数据进行更高层级分析。本工作证实了这些技术在运行环境中的可实施性。