The recently developed Prior-Data Fitted Networks (PFNs) have shown very promising results for applications in low-data regimes. The TabPFN model, a special case of PFNs for tabular data, is able to achieve state-of-the-art performance on a variety of classification tasks while producing posterior predictive distributions in mere seconds by in-context learning without the need for learning parameters or hyperparameter tuning. This makes TabPFN a very attractive option for a wide range of domain applications. However, a major drawback of the method is its lack of interpretability. Therefore, we propose several adaptations of popular interpretability methods that we specifically design for TabPFN. By taking advantage of the unique properties of the model, our adaptations allow for more efficient computations than existing implementations. In particular, we show how in-context learning facilitates the estimation of Shapley values by avoiding approximate retraining and enables the use of Leave-One-Covariate-Out (LOCO) even when working with large-scale Transformers. In addition, we demonstrate how data valuation methods can be used to address scalability challenges of TabPFN. Our proposed methods are implemented in a package tabpfn_iml and made available at https://github.com/david-rundel/tabpfn_iml.

翻译：近期发展的先验数据拟合网络（PFN）在低数据场景应用中展现出极具前景的结果。面向表格数据的TabPFN模型作为PFN的特殊情况，能够通过上下文学习在无需学习参数或超参数调优的情况下，在数秒内生成后验预测分布，并在多种分类任务中达到最先进性能。这使得TabPFN成为广泛领域应用中极具吸引力的选择。然而，该方法的主要缺陷在于缺乏可解释性。为此，我们提出针对TabPFN专门设计的多种流行可解释性方法适配方案。通过利用该模型的独特性质，我们的适配方案相比现有实现能够实现更高效的计算。特别地，我们展示了上下文学习如何通过避免近似重训练来促进沙普利值的估计，并使"留一协变量法（LOCO）"在处理大规模Transformer时依然可行。此外，我们论证了数据估值方法如何用于解决TabPFN的可扩展性挑战。所提出的方法已在tabpfn_iml工具包中实现，并发布于https://github.com/david-rundel/tabpfn_iml。