Graphs are central to modeling relationships in scientific computing, data analysis, and AI/ML, but their growing scale can exceed the memory and compute capacity of single nodes, requiring distributed solutions. Existing distributed graph framework, however, face fundamental challenges: graph algorithms are latency-bound, suffer from irregular memory access, and often impose synchronization costs that limit scalability and efficiency. In this work, we present a distributed implementation of the NWGraph library integrated with the HPX runtime system. By leveraging HPX's asynchronous many-task model, our approach aims to reduce synchronization overhead, improve load balance, and provide a foundation for distributed graph analytics. We evaluate this approach using two representative algorithms: Breadth-First-Search (BFS) and (PageRank). Our initial results show that BFS achieves better performance than the distributed Boost Graph Library (BGL), while PageRank remains more challenging, with current implementation not yet outperforming BGL. These findings highlight both the promise and the open challenges of applying asynchronous task-based runtimes to graph processing, and point to opportunities for future optimizations and extensions.
翻译:图在科学计算、数据分析和人工智能/机器学习中对关系建模至关重要,但其日益增长的规模可能超出单节点的内存与计算能力,从而需要分布式解决方案。然而,现有的分布式图框架面临若干根本性挑战:图算法受限于通信延迟,存在不规则内存访问,并且常常引入同步开销,从而限制了可扩展性与效率。本文提出一种与HPX运行时系统集成的NWGraph库的分布式实现。通过利用HPX的异步多任务模型,我们的方法旨在降低同步开销、改善负载均衡,并为分布式图分析提供基础。我们使用两种代表性算法——广度优先搜索(BFS)与PageRank——对该方法进行评估。初步结果表明,BFS的性能优于分布式Boost图库(BGL),而PageRank的实现仍更具挑战性,当前版本尚未超越BGL。这些发现既揭示了基于异步任务运行时应用于图处理的前景,也指出了其面临的开放挑战,同时为未来的优化与扩展指明了方向。