Sparse matrix computations are ubiquitous in scientific computing. With the recent interest in scientific machine learning, it is natural to ask how sparse matrix computations can leverage neural networks (NN). Unfortunately, multi-layer perceptron (MLP) neural networks are typically not natural for either graph or sparse matrix computations. The issue lies with the fact that MLPs require fixed-sized inputs while scientific applications generally generate sparse matrices with arbitrary dimensions and a wide range of nonzero patterns (or matrix graph vertex interconnections). While convolutional NNs could possibly address matrix graphs where all vertices have the same number of nearest neighbors, a more general approach is needed for arbitrary sparse matrices, e.g. arising from discretized partial differential equations on unstructured meshes. Graph neural networks (GNNs) are one approach suitable to sparse matrices. GNNs define aggregation functions (e.g., summations) that operate on variable size input data to produce data of a fixed output size so that MLPs can be applied. The goal of this paper is to provide an introduction to GNNs for a numerical linear algebra audience. Concrete examples are provided to illustrate how many common linear algebra tasks can be accomplished using GNNs. We focus on iterative methods that employ computational kernels such as matrix-vector products, interpolation, relaxation methods, and strength-of-connection measures. Our GNN examples include cases where parameters are determined a-priori as well as cases where parameters must be learned. The intent with this article is to help computational scientists understand how GNNs can be used to adapt machine learning concepts to computational tasks associated with sparse matrices. It is hoped that this understanding will stimulate data-driven extensions of classical sparse linear algebra tasks.

翻译：稀疏矩阵计算在科学计算领域无处不在。随着科学机器学习领域的兴起，人们自然要问：稀疏矩阵计算如何利用神经网络？遗憾的是，多层感知器神经网络通常不适用于图计算或稀疏矩阵计算。问题在于 MLP 需要固定大小的输入，而科学应用通常会产生具有任意维度和各种非零模式（或矩阵图顶点互连）的稀疏矩阵。虽然卷积神经网络可能适用于所有顶点具有相同最近邻数量的矩阵图，但对于任意稀疏矩阵（例如非结构化网格上离散化偏微分方程产生的矩阵），需要更通用的方法。图神经网络是适用于稀疏矩阵的一种方法。GNN 定义了聚合函数（例如求和），这些函数对可变大小的输入数据进行操作，以生成固定大小的输出数据，从而可以应用 MLP。本文旨在为数值线性代数领域的读者介绍 GNN。我们提供了具体示例来说明如何使用 GNN 完成许多常见的线性代数任务。我们重点关注采用矩阵-向量乘积、插值、松弛方法和连接强度度量等计算核心的迭代方法。我们的 GNN 示例包括参数预先确定的情况以及必须学习参数的情况。本文旨在帮助计算科学家理解如何将 GNN 用于将机器学习概念应用于与稀疏矩阵相关的计算任务。希望这种理解能够激发经典稀疏线性代数任务的数据驱动扩展。