We introduce StarDist, a Domain Specific Language for generating high-performant distributed graph algorithms in the message passing model. Our analysis-transformation framework optimizes graph traversal based on graph property access patterns, reduces global lock acquisitions on distributed structures, and minimizes message queues used in reduction operations. We provide a network optimized communication runtime for reduction operations that couples with our analysis framework and optimizes the propagation of updates based on vertex residency. StarDist is able to identify monotonic reduction blocks and is able to fuse reduction iterations over graphs into \textit{pulses}. We evaluate StarDist using three fundamental graph algorithms belonging to the CONGEST model: single-source shortest paths, weakly connected components, and PageRank computation, using a suite comprising both real-world and synthetic graphs across varying densities of topological compaction. Our results illustrate that the code generated with StarDist outperforms the distributed frameworks DRONE and D-Galois by an average of 19$\times$ and 7$\times$, respectively on our high communication setup and by 1.4$\times$ and 1.92$\times$ respectively on our high congestion network setup when averaged across all three algorithms.
翻译:我们提出StarDist,一种用于在消息传递模型中生成高性能分布式图算法的领域特定语言。我们的分析-变换框架基于图属性访问模式优化图遍历,减少分布式结构上的全局锁获取次数,并最小化归约操作中使用的消息队列。我们为归约操作提供了一种网络优化的通信运行时,该运行时与我们的分析框架耦合,并根据顶点驻留性优化更新的传播。StarDist能够识别单调归约块,并将图上的归约迭代融合为\textit{脉冲}。我们使用属于CONGEST模型的三种基本图算法评估StarDist:单源最短路径、弱连通分量和PageRank计算,测试集包含不同拓扑压缩密度的真实世界图和合成图。结果表明,在三种算法平均下,我们生成的高通信配置代码分别比分布式框架DRONE和D-Galois平均快19倍和7倍,在高拥塞网络配置下分别快1.4倍和1.92倍。