Graph Neural Network (GNN) inference is used in many real-world applications. Data sparsity in GNN inference, including sparsity in the input graph and the GNN model, offer opportunities to further speed up inference. Also, many pruning techniques have been proposed for model compression that increase the data sparsity of GNNs. We propose Dynasparse, a comprehensive hardware-software codesign on FPGA to accelerate GNN inference through dynamic sparsity exploitation. For this, we decouple the GNN computation kernels from the basic computation primitives, and explore hardware-software codesign as follows: 1) Hardware design: We propose a novel unified accelerator design on FPGA to efficiently execute various computation primitives. We develop a customized soft processor that is tightly coupled with the accelerator to execute a runtime system. Moreover, we develop efficient hardware mechanisms to profile the data sparsity and perform on-the-fly data format transformation to prepare the input data for various computation primitives; 2) Software design: We develop a runtime system that works synergistically with the accelerator to perform dynamic kernel-to-primitive mapping based on data sparsity. We implement Dynasparse on a state-of-the-art FPGA platform, Xilinx Alveo U250, and evaluate the design using widely used GNN models (GCN, GraphSAGE, GIN and SGC). For the above GNN models and various input graphs, the proposed accelerator and dynamic kernel-to-primitive mapping reduces the inference latency by $3.73\times$ on the average compared with the static mapping strategies employed in the state-of-the-art GNN accelerators. Compared with state-of-the-art CPU (GPU) implementations, Dynasparse achieves up to $56.9\times$ ($2.37\times$) speedup in end-to-end latency.

翻译：图神经网络（GNN）推理广泛应用于众多实际场景中。GNN推理中的数据稀疏性（包括输入图稀疏性和GNN模型稀疏性）为进一步加速推理提供了机会。此外，已有多种剪枝技术通过增加GNN的数据稀疏性实现模型压缩。本文提出Dynasparse，一种基于FPGA的软硬件协同设计方案，通过动态稀疏性利用来加速GNN推理。为此，我们将GNN计算内核解耦为基本计算原语，并探索以下软硬件协同设计：1）硬件设计：我们提出一种新颖的FPGA统一加速器架构，高效执行各类计算原语；开发了与加速器紧耦合的定制化软处理器以执行运行时系统；同时设计了高效硬件机制来剖析数据稀疏性，并执行在线数据格式转换，为不同计算原语准备输入数据；2）软件设计：我们开发了一套运行时系统，与加速器协同工作，基于数据稀疏性实现动态的核到原语映射。在业界领先的FPGA平台Xilinx Alveo U250上实现了Dynasparse，并使用广泛采用的GNN模型（GCN、GraphSAGE、GIN和SGC）进行评估。对于上述GNN模型及多种输入图，与现有最优GNN加速器中采用的静态映射策略相比，所提出的加速器及动态核到原语映射平均将推理延迟降低至1/3.73。与现有最优CPU（GPU）实现相比，Dynasparse在端到端延迟上实现了最高56.9倍（2.37倍）的加速。