Neural network models often struggle with high-dimensional but small sample-size tabular datasets. One reason is that current weight initialisation methods assume independence between weights, which can be problematic when there are insufficient samples to estimate the model's parameters accurately. In such small data scenarios, leveraging additional structures can improve the model's performance and training stability. To address this, we propose GCondNet, a general approach to enhance neural networks by leveraging implicit structures present in tabular data. We create a graph between samples for each data dimension, and utilise Graph Neural Networks (GNNs) for extracting this implicit structure, and for conditioning the parameters of the first layer of an underlying predictor network. By creating many small graphs, GCondNet exploits the data's high-dimensionality, and thus improves the performance of an underlying predictor network. We demonstrate the effectiveness of our method on 9 real-world datasets, where GCondNet outperforms 15 standard and state-of-the-art methods. The results show that GCondNet is a versatile framework for injecting graph-regularisation into various types of neural networks, including MLPs and tabular Transformers.

翻译：神经网络模型在处理高维但小样本的表格数据集时常常面临挑战。其中一个原因是当前的权重初始化方法假设权重之间相互独立，这在样本量不足以准确估计模型参数时会引发问题。在这种小样本场景中，利用额外结构可以提升模型性能与训练稳定性。为此，我们提出GCondNet——一种通过利用表格数据中隐含结构来增强神经网络的通用方法。我们针对每个数据维度构建样本间的图，并利用图神经网络（GNNs）提取这种隐含结构，进而对底层预测网络首层参数进行条件化处理。通过构建大量小型图，GCondNet充分利用了数据的高维特性，从而提升了底层预测网络的性能。我们在9个真实数据集上验证了该方法的有效性，结果显示GCondNet优于15种标准方法与最新技术。结果表明，GCondNet是一种可将图正则化注入各类神经网络（包括MLPs与表格Transformer）的通用框架。