Deep Learning has revolutionized the field of AI and led to remarkable achievements in applications involving image and text data. Unfortunately, there is inconclusive evidence on the merits of neural networks for structured tabular data. In this paper, we introduce a large-scale empirical study comparing neural networks against gradient-boosted decision trees on tabular data, but also transformer-based architectures against traditional multi-layer perceptrons (MLP) with residual connections. In contrast to prior work, our empirical findings indicate that neural networks are competitive against decision trees. Furthermore, we assess that transformer-based architectures do not outperform simpler variants of traditional MLP architectures on tabular datasets. As a result, this paper helps the research and practitioner communities make informed choices on deploying neural networks on future tabular data applications.

翻译：深度学习已彻底改变人工智能领域，并在图像和文本数据应用中取得了显著成就。然而，关于神经网络在结构化表格数据上的优势，目前尚未有确凿证据。本文开展了一项大规模实证研究，将神经网络与梯度提升决策树在表格数据上进行对比，同时比较了基于Transformer的架构与带残差连接的传统多层感知机（MLP）。与先前研究不同，我们的实证结果表明神经网络与决策树具有竞争力。此外，我们评估发现基于Transformer的架构在表格数据集上并未优于传统MLP架构的简化变体。因此，本文有助于研究人员和从业者群体在未来表格数据应用中做出部署神经网络的知情选择。