Successful machine learning on graphs or networks requires embeddings that not only represent nodes and edges as low-dimensional vectors but also preserve the graph structure. Established methods for generating embeddings require flexible exploration of the entire graph through repeated use of random walks that capture graph structure with samples of nodes and edges. These methods create scalability challenges for massive graphs with millions-to-billions of edges because single-node solutions have inadequate memory and processing capabilities. We present NOMAD, a distributed-memory graph embedding framework using the Message Passing Interface (MPI) for distributed graphs. NOMAD implements proximity-based models proposed in the widely popular LINE (Large-scale Information Network Embedding) algorithm. We propose several practical trade-offs to improve the scalability and communication overheads confronted by irregular and distributed graph embedding methods, catering to massive-scale graphs arising in web and science domains. NOMAD demonstrates median speedups of 10/100x on CPU-based NERSC Perlmutter cluster relative to the popular reference implementations of multi-threaded LINE and node2vec, 35-76x over distributed PBG, and competitive embedding quality relative to LINE, node2vec, and GraphVite, while yielding 12-370x end-to-end speedups on real-world graphs.

翻译：在图形或网络上的成功机器学习不仅需要将节点和边表示为低维向量，还需保留图结构。现有生成嵌入的方法通过重复使用随机游走来灵活探索整个图，这些随机游走通过节点和边的样本捕获图结构。这些方法在处理包含数百万至数十亿条边的大规模图时面临可扩展性挑战，因为单节点解决方案存在内存和处理能力不足的问题。我们提出NOMAD，一种基于消息传递接口（MPI）的分布式内存图嵌入框架，适用于分布式图。NOMAD实现了广泛流行的LINE（大规模信息网络嵌入）算法中提出的基于邻近度的模型。我们提出了若干实际权衡策略，以改善非规则和分布式图嵌入方法面临的可扩展性和通信开销问题，从而适应于网络和科学领域中产生的大规模图。在基于CPU的NERSC Perlmutter集群上，NOMAD相对于流行的多线程LINE和node2vec参考实现实现了10/100倍的中位加速，相对于分布式PBG实现了35-76倍加速，且相对于LINE、node2vec和GraphVite具有竞争力的嵌入质量，同时在真实世界图上实现了12-370倍的端到端加速。