Anderson acceleration (AA) is a well-known method for accelerating the convergence of iterative algorithms, with applications in various fields including deep learning and optimization. Despite its popularity in these areas, the effectiveness of AA in classical machine learning classifiers has not been thoroughly studied. Tabular data, in particular, presents a unique challenge for deep learning models, and classical machine learning models are known to perform better in these scenarios. However, the convergence analysis of these models has received limited attention. To address this gap in research, we implement a support vector machine (SVM) classifier variant that incorporates AA to speed up convergence. We evaluate the performance of our SVM with and without Anderson acceleration on several datasets from the biology domain and demonstrate that the use of AA significantly improves convergence and reduces the training loss as the number of iterations increases. Our findings provide a promising perspective on the potential of Anderson acceleration in the training of simple machine learning classifiers and underscore the importance of further research in this area. By showing the effectiveness of AA in this setting, we aim to inspire more studies that explore the applications of AA in classical machine learning.

翻译：安德森加速（AA）是一种广为人知的迭代算法收敛加速方法，在深度学习与优化等多个领域均有应用。尽管AA在这些领域广受欢迎，但其在经典机器学习分类器中的有效性尚未得到充分研究。特别是表格数据对深度学习模型构成独特挑战，而经典机器学习模型在此类场景中表现更优，然而这些模型的收敛性分析却鲜受关注。为填补这一研究空白，我们实现了一种融合AA以加速收敛的支持向量机（SVM）分类器变体。通过在生物学领域的多个数据集上评估有无安德森加速的SVM性能，我们证明使用AA能显著提升收敛速度，并随迭代次数增加有效降低训练损失。我们的发现为安德森加速在简单机器学习分类器训练中的应用提供了颇具前景的视角，并凸显了在该领域开展进一步研究的重要性。通过展示AA在此场景中的有效性，我们旨在激励更多探索AA在经典机器学习中应用的研究。