Recent tabular Foundational Models (FM) such as TabPFN and TabICL, leverage in-context learning to achieve strong performance without gradient updates or fine-tuning. However, their robustness to adversarial manipulation remains largely unexplored. In this work, we present a comprehensive study of the adversarial vulnerabilities of tabular FM, focusing on both their fragility to targeted test-time attacks and their potential misuse as adversarial tools. We show on three benchmarks in finance, cybersecurity and healthcare, that small, structured perturbations to test inputs can significantly degrade prediction accuracy, even when training context remain fixed. Additionally, we demonstrate that tabular FM can be repurposed to generate transferable evasion to conventional models such as random forests and XGBoost, and on a lesser extent to deep tabular models. To improve tabular FM, we formulate the robustification problem as an optimization of the weights (adversarial fine-tuning), or the context (adversarial in-context learning). We introduce an in-context adversarial training strategy that incrementally replaces the context with adversarial perturbed instances, without updating model weights. Our approach improves robustness across multiple tabular benchmarks. Together, these findings position tabular FM as both a target and a source of adversarial threats, highlighting the urgent need for robust training and evaluation practices in this emerging paradigm.

翻译：近期，TabPFN 和 TabICL 等表格基础模型利用上下文学习，在不进行梯度更新或微调的情况下取得了强劲的性能。然而，它们对对抗性操控的鲁棒性在很大程度上仍未得到探索。在本工作中，我们对表格基础模型的对抗性脆弱性进行了全面研究，重点关注其对定向测试时攻击的脆弱性以及被用作对抗性工具的潜在风险。我们在金融、网络安全和医疗保健三个基准测试上表明，即使训练上下文保持不变，对测试输入施加微小的结构性扰动也会显著降低预测准确度。此外，我们证明表格基础模型可被重新利用以生成对随机森林和 XGBoost 等传统模型具有迁移性的逃避攻击，在较小程度上也对深度表格模型有效。为了改进表格基础模型，我们将其鲁棒性增强问题表述为对权重（对抗性微调）或上下文（对抗性上下文学习）的优化。我们引入了一种上下文对抗性训练策略，该策略在不更新模型权重的情况下，逐步用对抗性扰动实例替换上下文。我们的方法在多个表格基准测试上提升了鲁棒性。综上，这些发现将表格基础模型定位为对抗性威胁的目标与来源，凸显了在这一新兴范式中进行鲁棒训练与评估实践的迫切需求。