Generalized Additive Models (GAMs) are widely recognized for their ability to create fully interpretable machine learning models for tabular data. Traditionally, training GAMs involves iterative learning algorithms, such as splines, boosted trees, or neural networks, which refine the additive components through repeated error reduction. In this paper, we introduce GAMformer, the first method to leverage in-context learning to estimate shape functions of a GAM in a single forward pass, representing a significant departure from the conventional iterative approaches to GAM fitting. Building on previous research applying in-context learning to tabular data, we exclusively use complex, synthetic data to train GAMformer, yet find it extrapolates well to real-world data. Our experiments show that GAMformer performs on par with other leading GAMs across various classification benchmarks while generating highly interpretable shape functions.

翻译：广义可加模型（GAMs）因其能为表格数据构建完全可解释的机器学习模型而广受认可。传统上，训练GAMs涉及迭代学习算法，例如样条、提升树或神经网络，这些方法通过反复减少误差来优化加性分量。本文提出GAMformer，这是首个利用上下文学习在单次前向传播中估计GAM形状函数的方法，标志着对传统GAM拟合迭代路径的重要突破。基于先前将上下文学习应用于表格数据的研究，我们仅使用复杂的合成数据训练GAMformer，但发现其能良好泛化至现实世界数据。实验表明，GAMformer在多种分类基准测试中与其他主流GAMs性能相当，同时能生成高度可解释的形状函数。