Data organized in tabular format is ubiquitous in real-world applications, and users often craft tables with biased feature definitions and flexibly set prediction targets of their interests. Thus, a rapid development of a robust, effective, dataset-versatile, user-friendly tabular prediction approach is highly desired. While Gradient Boosting Decision Trees (GBDTs) and existing deep neural networks (DNNs) have been extensively utilized by professional users, they present several challenges for casual users, particularly: (i) the dilemma of model selection due to their different dataset preferences, and (ii) the need for heavy hyperparameter searching, failing which their performances are deemed inadequate. In this paper, we delve into this question: Can we develop a deep learning model that serves as a "sure bet" solution for a wide range of tabular prediction tasks, while also being user-friendly for casual users? We delve into three key drawbacks of deep tabular models, encompassing: (P1) lack of rotational variance property, (P2) large data demand, and (P3) over-smooth solution. We propose ExcelFormer, addressing these challenges through a semi-permeable attention module that effectively constrains the influence of less informative features to break the DNNs' rotational invariance property (for P1), data augmentation approaches tailored for tabular data (for P2), and attentive feedforward network to boost the model fitting capability (for P3). These designs collectively make ExcelFormer a "sure bet" solution for diverse tabular datasets. Extensive and stratified experiments conducted on real-world datasets demonstrate that our model outperforms previous approaches across diverse tabular data prediction tasks, and this framework can be friendly to casual users, offering ease of use without the heavy hyperparameter tuning.

翻译：表格形式的数据在现实应用中无处不在，用户常常会构建带有偏置特征定义的表，并灵活地设置其感兴趣的预测目标。因此，亟需开发一种鲁棒、有效、数据集通用且用户友好的表格预测方法。尽管梯度提升决策树和现有的深度神经网络已被专业用户广泛使用，但它们对普通用户而言仍存在若干挑战，尤其是：（i）因其对不同数据集的偏好而导致的模型选择困境，以及（ii）需要进行繁重的超参数搜索，否则其性能被认为不足。在本文中，我们深入探讨了以下问题：能否开发一种深度学习模型，作为适用于广泛表格预测任务的“可靠选择”解决方案，同时对普通用户也保持友好？我们深入分析了深度表格模型的三个关键缺点，包括：（P1）缺乏旋转方差特性，（P2）数据需求量大，以及（P3）解过于平滑。我们提出了ExcelFormer，通过以下方式应对这些挑战：一个半渗透注意力模块，有效约束信息量较少的特征的影响，以打破深度神经网络的旋转不变性（针对P1）；为表格数据量身定制的数据增强方法（针对P2）；以及注意力前馈网络，以提升模型拟合能力（针对P3）。这些设计共同使ExcelFormer成为适用于多样化表格数据集的“可靠选择”解决方案。在真实世界数据集上进行的大量分层实验表明，我们的模型在多种表格数据预测任务上优于先前的方法，并且该框架对普通用户友好，易于使用，无需繁重的超参数调优。