Graph partitioning plays a pivotal role in various distributed graph processing applications, including graph analytics, graph neural network training, and distributed graph databases. Graphs that require distributed settings are often too large to fit in the main memory of a single machine. This challenge renders traditional in-memory graph partitioners infeasible, leading to the emergence of streaming solutions. Streaming partitioners produce lower-quality partitions because they work from partial information and must make premature decisions before they have a complete view of a vertex's neighborhood. We introduce CUTTANA, a streaming graph partitioner that partitions massive graphs (Web/Twitter scale) with superior quality compared to existing streaming solutions. CUTTANA uses a novel buffering technique that prevents the premature assignment of vertices to partitions and a scalable coarsening and refinement technique that enables a complete graph view, improving the intermediate assignment made by a streaming partitioner. We implemented a parallel version for CUTTANA that offers nearly the same partitioning latency as existing streaming partitioners. Our experimental analysis shows that CUTTANA consistently yields better partitioning quality than existing state-of-the-art streaming vertex partitioners in terms of both edge-cut and communication volume metrics. We also evaluate the workload latencies that result from using CUTTANA and other partitioners in distributed graph analytics and databases. CUTTANA outperforms the other methods in most scenarios (algorithms, datasets). In analytics applications, CUTTANA improves runtime performance by up to 59% compared to various streaming partitioners (HDRF, Fennel, Ginger, HeiStream). In graph database tasks, CUTTANA results in higher query throughput by up to 23%, without hurting tail latency.

翻译：图划分在分布式图处理应用中扮演关键角色，涵盖图分析、图神经网络训练以及分布式图数据库等领域。需要分布式环境的图通常规模过大，无法单机主存容纳。这一挑战使得传统内存图划分器不可行，从而催生了流式解决方案。流式划分器因基于局部信息运作，且需在未完整观测顶点邻域时做出过早决策，导致划分质量较低。我们提出CUTTANA——一种流式图划分器，其能够以优于现有流式方案的划分质量，对大规模图（如Web/Twitter规模）进行划分。CUTTANA采用新型缓冲技术避免顶点过早分配给分区，并引入可扩展的粗化与精化技术以实现完整图视角，从而改进流式划分器生成的中间分配结果。我们实现了CUTTANA并行版本，其划分延迟与现有流式划分器几乎持平。实验分析表明，在边割与通信量两类指标上，CUTTANA的划分质量始终优于当前最先进的流式顶点划分器。我们还评估了在分布式图分析与数据库中采用CUTTANA及其他划分器时的工作负载延迟。在大多数场景（算法、数据集）下，CUTTANA均优于其他方法。在图分析应用中，相较于多种流式划分器（如HDRF、Fennel、Ginger、HeiStream），CUTTANA可将运行时性能提升最高达59%。在图数据库任务中，CUTTANA将查询吞吐量提升最高达23%，且不损害尾延迟。