Covariance Neural Networks (VNNs) perform graph convolutions on the covariance matrix of tabular data and achieve success in a variety of applications. However, the empirical covariance matrix on which the VNNs operate may contain many spurious correlations, making VNNs' performance inconsistent due to these noisy estimates and decreasing their computational efficiency. To tackle this issue, we put forth Sparse coVariance Neural Networks (S-VNNs), a framework that applies sparsification techniques on the sample covariance matrix before convolution. When the true covariance matrix is sparse, we propose hard and soft thresholding to improve covariance estimation and reduce computational cost. Instead, when the true covariance is dense, we propose stochastic sparsification where data correlations are dropped in probability according to principled strategies. We show that S-VNNs are more stable than nominal VNNs as well as sparse principal component analysis. By analyzing the impact of sparsification on their behavior, we provide novel connections between S-VNN stability and data distribution. We support our theoretical findings with experimental results on various application scenarios, ranging from brain data to human action recognition, and show an improved task performance, stability, and computational efficiency of S-VNNs compared with nominal VNNs.

翻译：协方差神经网络（VNNs）在表格数据的协方差矩阵上执行图卷积操作，并在多种应用中取得成功。然而，VNNs所操作的样本协方差矩阵可能包含大量虚假相关性，这些噪声估计导致VNNs的性能不稳定，并降低了其计算效率。为解决此问题，我们提出了稀疏协方差神经网络（S-VNNs），该框架在卷积前对样本协方差矩阵应用稀疏化技术。当真实协方差矩阵稀疏时，我们提出硬阈值和软阈值方法以改进协方差估计并降低计算成本。反之，当真实协方差稠密时，我们提出随机稀疏化方法，其中数据相关性根据有原则的策略以概率方式被丢弃。我们证明，S-VNNs比标准VNNs以及稀疏主成分分析更为稳定。通过分析稀疏化对其行为的影响，我们建立了S-VNN稳定性与数据分布之间的新联系。我们在从脑数据到人类动作识别的多种应用场景中提供了实验结果以支持理论发现，并展示了S-VNNs相较于标准VNNs在任务性能、稳定性和计算效率方面的提升。