Anderson acceleration (AA) is a well-known method for accelerating the convergence of iterative algorithms, with applications in various fields including deep learning and optimization. Despite its popularity in these areas, the effectiveness of AA in classical machine learning classifiers has not been thoroughly studied. Tabular data, in particular, presents a unique challenge for deep learning models, and classical machine learning models are known to perform better in these scenarios. However, the convergence analysis of these models has received limited attention. To address this gap in research, we implement a support vector machine (SVM) classifier variant that incorporates AA to speed up convergence. We evaluate the performance of our SVM with and without Anderson acceleration on several datasets from the biology domain and demonstrate that the use of AA significantly improves convergence and reduces the training loss as the number of iterations increases. Our findings provide a promising perspective on the potential of Anderson acceleration in the training of simple machine learning classifiers and underscore the importance of further research in this area. By showing the effectiveness of AA in this setting, we aim to inspire more studies that explore the applications of AA in classical machine learning.

翻译：安德森加速（AA）是一种广为人知的迭代算法收敛加速技术，在深度学习和优化等众多领域均有应用。尽管AA在这些领域广受欢迎，但其在经典机器学习分类器中的有效性尚未得到充分研究。特别是表格数据为深度学习模型带来了独特挑战，而经典机器学习模型在此类场景中表现更佳。然而，这些模型的收敛性分析却鲜受关注。为填补这一研究空白，我们实现了一种支持向量机（SVM）分类器变体，该变体融合了AA以加速收敛。我们基于多个生物学领域数据集，对采用与未采用安德森加速的SVM性能进行了评估，结果表明，随着迭代次数增加，AA的使用显著提升了收敛速度并降低了训练损失。我们的发现为安德森加速在简单机器学习分类器训练中的应用前景提供了新视角，并凸显了在该领域深化研究的重要性。通过展示AA在此场景中的有效性，我们旨在激发更多探索AA在经典机器学习中应用的研究。