Tabular foundation models (TFMs) such as TabPFN (Tabular Prior-Data Fitted Network) are designed to generalize across heterogeneous tabular datasets through in-context learning (ICL). They perform prediction in a single forward pass conditioned on labeled examples without dataset-specific parameter updates. This paradigm is particularly attractive in industrial domains (e.g., finance and healthcare) where tabular prediction is pervasive. Retraining a bespoke model for each new table can be costly or infeasible in these settings, while data quality issues such as irrelevant predictors, correlated feature groups, and label noise are common. In this paper, we provide strong empirical evidence that TabPFN is highly robust under these sub-optimal conditions. We study TabPFN and its attention mechanisms for binary classification problems with controlled synthetic perturbations that vary: (i) dataset width by injecting random uncorrelated features and by introducing nonlinearly correlated features, (ii) dataset size by increasing the number of training rows, and (iii) label quality by increasing the fraction of mislabeled targets. Beyond predictive performance, we analyze internal signals including attention concentration and attention-based feature ranking metrics. Across these parametric tests, TabPFN is remarkably resilient: ROC-AUC remains high, attention stays structured and sharp, and informative features are highly ranked by attention-based metrics. Qualitative visualizations with attention heatmaps, feature-token embeddings, and SHAP plots further support a consistent pattern across layers in which TabPFN increasingly concentrates on useful features while separating their signals from noise. Together, these findings suggest that TabPFN is a robust TFM capable of maintaining both predictive performance and coherent internal behavior under various scenarios of data imperfections.

翻译：表格式基础模型（如TabPFN）旨在通过上下文学习（ICL）泛化到异构表格数据集。它们通过单次前向传播，基于标注样本进行预测，无需针对特定数据集更新参数。这种范式在表格预测普遍存在的工业领域（如金融和医疗）中尤为吸引人。在这些场景中，为每个新表重新训练定制模型可能成本高昂或不可行，而数据质量问题（如无关预测因子、相关特征组和标签噪声）却很常见。本文通过强有力的实证证据表明，TabPFN在这些非理想条件下具有高度鲁棒性。我们针对二分类问题，通过受控合成扰动研究TabPFN及其注意力机制，这些扰动包括：（i）通过注入随机无关特征和引入非线性相关特征改变数据集宽度，（ii）通过增加训练样本行数改变数据集规模，（iii）通过增加错误标注样本比例改变标签质量。除了预测性能，我们还分析了内部信号，包括注意力集中度和基于注意力的特征排序指标。在这些参数化测试中，TabPFN表现出显著韧性：ROC-AUC保持较高水平，注意力保持结构化和聚焦性，且基于注意力的指标对信息性特征给予高排序。通过注意力热图、特征-令牌嵌入和SHAP图进行的定性可视化进一步支持跨层的一致模式：TabPFN越来越集中于有用特征，同时将其信号与噪声分离。综合这些发现表明，TabPFN是一种鲁棒的表格式基础模型，能够在各种数据不完备场景下同时保持预测性能和连贯的内部行为。