Algorithm selection, a critical process of automated machine learning, aims to identify the most suitable algorithm for solving a specific problem prior to execution. Mainstream algorithm selection techniques heavily rely on problem features, while the role of algorithm features remains largely unexplored. Due to the intrinsic complexity of algorithms, effective methods for universally extracting algorithm information are lacking. This paper takes a significant step towards bridging this gap by introducing Large Language Models (LLMs) into algorithm selection for the first time. By comprehending the code text, LLM not only captures the structural and semantic aspects of the algorithm, but also demonstrates contextual awareness and library function understanding. The high-dimensional algorithm representation extracted by LLM, after undergoing a feature selection module, is combined with the problem representation and passed to the similarity calculation module. The selected algorithm is determined by the matching degree between a given problem and different algorithms. Extensive experiments validate the performance superiority of the proposed model and the efficacy of each key module. Furthermore, we present a theoretical upper bound on model complexity, showcasing the influence of algorithm representation and feature selection modules. This provides valuable theoretical guidance for the practical implementation of our method.

翻译：算法选择作为自动化机器学习中的关键过程，旨在在执行前识别出最适合解决特定问题的算法。主流算法选择技术高度依赖问题特征，而算法特征的作用在很大程度上尚未被探索。由于算法固有的复杂性，缺乏用于普遍提取算法信息的有效方法。本文通过首次将大型语言模型（LLMs）引入算法选择，迈出了弥补这一差距的重要一步。通过理解代码文本，LLM不仅捕捉算法的结构和语义方面，还展现出上下文感知和库函数理解能力。由LLM提取的高维算法表示在经过特征选择模块处理后，与问题表示相结合，并传递给相似度计算模块。所选算法由给定问题与不同算法之间的匹配程度决定。广泛实验验证了所提出模型的性能优越性以及每个关键模块的有效性。此外，我们提出了模型复杂度的理论上限，展示了算法表示和特征选择模块的影响。这为我们的方法在实际实施中提供了宝贵的理论指导。