Graph neural networks (GNNs) have recently empowered various novel computer vision (CV) tasks. In GNN-based CV tasks, a combination of CNN layers and GNN layers or only GNN layers are employed. This paper introduces GCV-Turbo, a domain-specific accelerator on FPGA for end-to-end acceleration of GNN-based CV tasks. GCV-Turbo consists of two key components: (1) a \emph{novel} hardware architecture optimized for the computation kernels in both CNNs and GNNs using the same set of computation resources. (2) a PyTorch-compatible compiler that takes a user-defined model as input, performs end-to-end optimization for the computation graph of a given GNN-based CV task, and produces optimized code for hardware execution. The hardware architecture and the compiler work synergistically to support a variety of GNN-based CV tasks. We implement GCV-Turbo on a state-of-the-art FPGA and evaluate its performance across six representative GNN-based CV tasks with diverse input data modalities (e.g., image, human skeleton, point cloud). Compared with state-of-the-art CPU (GPU) implementations, GCV-Turbo achieves an average latency reduction of $68.4\times$ ($4.1\times$) on these six GNN-based CV tasks. Moreover, GCV-Turbo supports the execution of the standalone CNNs or GNNs, achieving performance comparable to that of state-of-the-art CNN (GNN) accelerators for widely used CNN-only (GNN-only) models.

翻译：图神经网络（GNN）近期推动了多种新型计算机视觉（CV）任务的发展。在基于GNN的CV任务中，模型采用CNN层与GNN层组合或仅包含GNN层。本文提出GCV-Turbo，一种面向FPGA的领域专用加速器，用于实现基于GNN的CV任务的端到端加速。GCV-Turbo包含两个关键组件：（1）一种新型硬件架构，利用同一组计算资源优化CNN和GNN中的计算核心；（2）一个兼容PyTorch的编译器，以用户定义模型为输入，对给定GNN-CV任务的计算图进行端到端优化，并生成硬件执行优化代码。硬件架构与编译器协同工作，支持多种基于GNN的CV任务。我们在最先进的FPGA上实现GCV-Turbo，并在六种代表性GNN-CV任务中（涵盖图像、人体骨架、点云等不同输入数据模态）评估其性能。与最先进的CPU（GPU）实现相比，GCV-Turbo在六种GNN-CV任务上平均延迟降低68.4倍（4.1倍）。此外，GCV-Turbo支持独立CNN或GNN的执行，在广泛使用的纯CNN（纯GNN）模型上，其性能与最先进的CNN（GNN）加速器相当。