Tabular data is prevalent in real-world machine learning applications, and new models for supervised learning of tabular data are frequently proposed. Comparative studies assessing the performance of models typically consist of model-centric evaluation setups with overly standardized data preprocessing. This paper demonstrates that such model-centric evaluations are biased, as real-world modeling pipelines often require dataset-specific preprocessing and feature engineering. Therefore, we propose a data-centric evaluation framework. We select 10 relevant datasets from Kaggle competitions and implement expert-level preprocessing pipelines for each dataset. We conduct experiments with different preprocessing pipelines and hyperparameter optimization (HPO) regimes to quantify the impact of model selection, HPO, feature engineering, and test-time adaptation. Our main findings are: 1. After dataset-specific feature engineering, model rankings change considerably, performance differences decrease, and the importance of model selection reduces. 2. Recent models, despite their measurable progress, still significantly benefit from manual feature engineering. This holds true for both tree-based models and neural networks. 3. While tabular data is typically considered static, samples are often collected over time, and adapting to distribution shifts can be important even in supposedly static data. These insights suggest that research efforts should be directed toward a data-centric perspective, acknowledging that tabular data requires feature engineering and often exhibits temporal characteristics.

翻译：表格数据在现实世界的机器学习应用中普遍存在，针对表格数据监督学习的新模型也频繁被提出。评估模型性能的比较研究通常采用模型中心的评估设置，并伴随过度标准化的数据预处理。本文论证此类模型中心评估存在偏差，因为现实世界的建模流程通常需要数据集特定的预处理和特征工程。因此，我们提出了一个数据中心的评估框架。我们从Kaggle竞赛中选取了10个相关数据集，并为每个数据集实现了专家级的预处理流程。我们通过不同的预处理流程和超参数优化（HPO）方案进行实验，以量化模型选择、HPO、特征工程和测试时适应的影响。我们的主要发现如下：1. 经过数据集特定的特征工程后，模型排名发生显著变化，性能差异减小，模型选择的重要性降低。2. 近期模型尽管取得了可衡量的进展，但仍显著受益于手动特征工程。这一结论对基于树的模型和神经网络均成立。3. 尽管表格数据通常被视为静态数据，但样本往往随时间收集，即使在所谓的静态数据中，适应分布偏移也可能至关重要。这些见解表明，研究努力应转向以数据为中心的视角，承认表格数据需要特征工程，并常常表现出时间特性。