Recent advancements in large language models (LLMs) have shown remarkable potential in automating machine learning tasks. However, existing LLM-based agents often struggle with low-diversity and suboptimal code generation. While recent work has introduced Monte Carlo Tree Search (MCTS) to address these issues, limitations persist in the quality and diversity of thoughts generated, as well as in the scalar value feedback mechanisms used for node selection. In this study, we introduce Introspective Monte Carlo Tree Search (I-MCTS), a novel approach that iteratively expands tree nodes through an introspective process that meticulously analyzes solutions and results from parent and sibling nodes. This facilitates a continuous refinement of the node in the search tree, thereby enhancing the overall decision-making process. Furthermore, we integrate a Large Language Model (LLM)-based value model to facilitate direct evaluation of each node's solution prior to conducting comprehensive computational rollouts. A hybrid rewarding mechanism is implemented to seamlessly transition the Q-value from LLM-estimated scores to actual performance scores. This allows higher-quality nodes to be traversed earlier. Applied to the various ML tasks, our approach demonstrates a 4% absolute improvement in performance compared to the strong open-source AutoML agents, showcasing its effectiveness in enhancing agentic AutoML systems. Resource available at https://github.com/jokieleung/I-MCTS

翻译：大型语言模型（LLM）的最新进展在自动化机器学习任务方面展现出显著潜力。然而，现有的基于LLM的智能体常面临生成代码多样性不足和次优的问题。尽管近期研究引入蒙特卡洛树搜索（MCTS）以应对这些挑战，但在生成思维的质量与多样性以及节点选择的标量值反馈机制方面仍存在局限。本研究提出内省蒙特卡洛树搜索（I-MCTS），该方法通过内省过程迭代扩展树节点，该过程会细致分析父节点与兄弟节点的解决方案及结果。这种机制实现了搜索树中节点的持续优化，从而提升整体决策质量。此外，我们整合了基于大型语言模型（LLM）的价值模型，以便在开展全面计算推演前直接评估各节点的解决方案。通过实施混合奖励机制，将Q值从LLM预估分数平滑过渡至实际性能分数，使更高质量的节点得以优先遍历。在多种机器学习任务上的实验表明，相较于现有强大的开源AutoML智能体，本方法实现了4%的绝对性能提升，验证了其在增强自主AutoML系统方面的有效性。相关资源详见https://github.com/jokieleung/I-MCTS