The existing literature on deep learning for tabular data proposes a wide range of novel architectures and reports competitive results on various datasets. However, the proposed models are usually not properly compared to each other and existing works often use different benchmarks and experiment protocols. As a result, it is unclear for both researchers and practitioners what models perform best. Additionally, the field still lacks effective baselines, that is, the easy-to-use models that provide competitive performance across different problems. In this work, we perform an overview of the main families of DL architectures for tabular data and raise the bar of baselines in tabular DL by identifying two simple and powerful deep architectures. The first one is a ResNet-like architecture which turns out to be a strong baseline that is often missing in prior works. The second model is our simple adaptation of the Transformer architecture for tabular data, which outperforms other solutions on most tasks. Both models are compared to many existing architectures on a diverse set of tasks under the same training and tuning protocols. We also compare the best DL models with Gradient Boosted Decision Trees and conclude that there is still no universally superior solution.

翻译：现有的关于表格数据深度学习的文献提出了多种新架构，并在各种数据集上报告了具有竞争力的结果。然而，所提出的模型通常缺乏相互之间的适当比较，且现有工作常使用不同的基准和实验方案。因此，研究人员和从业者均难以明确哪些模型性能最佳。此外，该领域仍缺少有效的基线模型，即易于使用且能跨不同问题提供竞争性能的模型。本研究对表格数据的主要深度学习架构家族进行了综述，并通过识别两种简单而强大的深度架构，提升了表格深度学习基线的整体水平。第一种架构类似ResNet，事实证明，它是一个强大的基线，但在以往的工作中常常缺失。第二种模型是我们对Transformer架构针对表格数据的简单适配，在大多数任务上优于其他解决方案。我们将这两种模型与多种现有架构在相同训练和调优方案下，于多样化的任务上进行了比较。同时，我们也对比了最佳深度学习模型与梯度提升决策树的性能，并得出结论：目前仍不存在普遍优越的解决方案。