Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and losses. Extensive research has introduced new modeling techniques, complemented by large-scale benchmarking studies consolidating the state-of-the-art. Today, quasi-standards such as gradient-boosting models paired with SHAP explainers have emerged, yet continuous improvement of risk models remains a top priority. Concurrently, rapid advancements in AI, most notably large language models, have disrupted predictive modeling paradigms. Foundation models, pretrained on extensive datasets from diverse domains, have demonstrated remarkable performance by leveraging prior knowledge. While prevalent in natural language processing and computer vision, foundation models for tabular data have only recently emerged. We conjecture that pretraining on out-of-domain data is particularly beneficial in small-data settings, such as SME lending or specialized corporate portfolios, and may help address longstanding challenges including low default portfolios and class imbalance. This paper benchmarks recently proposed tabular foundation models against a broad set of competitors, including established and advanced machine learning techniques, across two core tasks: PD and LGD modeling. Our evaluation encompasses various datasets, performance indicators, and experimental conditions. We find that tabular foundation models generally perform best across datasets and tasks. Moreover, they offer significant improvement in predictive performance as dataset size shrinks. These results are remarkable given that the models are tested out-of-the-box, without hyperparameter tuning, ensuring ease of use and mitigating computational costs.

翻译：预测模型在信用风险管理中扮演着关键角色，通过精确估计违约概率和损失来指导重要决策。大量研究引入了新的建模技术，并辅以整合当前最优方法的大规模基准测试研究。如今，梯度提升模型结合SHAP解释器已成为准标准，但风险模型的持续改进仍是首要任务。与此同时，人工智能的快速发展，尤其是大语言模型，已颠覆了预测建模范式。基础模型通过在海量跨领域数据集上进行预训练，凭借利用先验知识展现出卓越性能。虽然基础模型在自然语言处理和计算机视觉领域已广泛应用，但针对表格数据的基础模型直到近期才出现。我们推测，在小样本数据场景（如中小企业贷款或专业企业组合）中，跨领域数据的预训练尤为有效，并可能帮助解决低违约组合和类别不平衡等长期挑战。本文对近期提出的表格基础模型进行了基准测试，将其与包括传统和先进机器学习技术在内的广泛对手进行了比较，涵盖PD和LGD建模两大核心任务。我们的评估涉及多种数据集、性能指标和实验条件。研究发现，表格基础模型在不同数据集和任务中普遍表现最佳。此外，随着数据集规模缩小，它们在预测性能上提供了显著提升。这些结果尤为引人注目，因为模型未经超参数调优即可直接使用，既确保了易用性又降低了计算成本。