The existing literature on deep learning for tabular data proposes a wide range of novel architectures and reports competitive results on various datasets. However, the proposed models are usually not properly compared to each other and existing works often use different benchmarks and experiment protocols. As a result, it is unclear for both researchers and practitioners what models perform best. Additionally, the field still lacks effective baselines, that is, the easy-to-use models that provide competitive performance across different problems. In this work, we perform an overview of the main families of DL architectures for tabular data and raise the bar of baselines in tabular DL by identifying two simple and powerful deep architectures. The first one is a ResNet-like architecture which turns out to be a strong baseline that is often missing in prior works. The second model is our simple adaptation of the Transformer architecture for tabular data, which outperforms other solutions on most tasks. Both models are compared to many existing architectures on a diverse set of tasks under the same training and tuning protocols. We also compare the best DL models with Gradient Boosted Decision Trees and conclude that there is still no universally superior solution.

翻译：现有关于表格数据深度学习的文献提出了多种新颖架构，并在各类数据集上取得了有竞争力的结果。然而，这些模型之间通常缺乏恰当的比较，且现有工作常采用不同的基准和实验协议。因此，研究者和实践者均难以明确哪种模型表现最佳。此外，该领域仍缺乏有效的基线方法，即那些能在不同问题上提供稳定竞争力的简易模型。本研究对表格数据的主要深度学习架构家族进行了综述，并通过识别两种简单而强大的深度架构，提升了表格深度学习基线的标准。第一种是类似ResNet的架构，它被证明是先前工作中常被忽视的强基线模型。第二种是我们针对表格数据对Transformer架构进行的简单适配，该模型在大多数任务上优于其他解决方案。在统一训练和调优协议下，我们将这两种模型与众多现有架构在多样化任务集上进行了比较。同时，我们还将最佳深度学习模型与梯度提升决策树进行了对比，结果表明目前仍不存在普遍优越的解决方案。