The field of machine learning (ML) has gained widespread adoption, leading to a significant demand for adapting ML to specific scenarios, which is yet expensive and non-trivial. The predominant approaches towards the automation of solving ML tasks (e.g., AutoML) are often time consuming and hard to understand for human developers. In contrast, though human engineers have the incredible ability to understand tasks and reason about solutions, their experience and knowledge are often sparse and difficult to utilize by quantitative approaches. In this paper, we aim to bridge the gap between machine intelligence and human knowledge by introducing a novel framework MLCopilot, which leverages the state-of-the-art LLMs to develop ML solutions for novel tasks. We showcase the possibility of extending the capability of LLMs to comprehend structured inputs and perform thorough reasoning for solving novel ML tasks. And we find that, after some dedicated design, the LLM can (i) observe from the existing experiences of ML tasks and (ii) reason effectively to deliver promising results for new tasks. The solution generated can be used directly to achieve high levels of competitiveness.

翻译：机器学习（ML）领域已广泛普及，导致对将ML适配至特定场景的需求显著增长，然而这一过程既昂贵又具有挑战性。当前自动化解决ML任务的主流方法（如AutoML）通常耗时且难以被人类开发者理解。相比之下，尽管人类工程师具有理解任务和推理解决方案的非凡能力，但其经验和知识往往稀疏且难以通过定量方法加以利用。本文旨在通过引入新型框架MLCopilot来弥合机器智能与人类知识之间的鸿沟，该框架利用最先进的LLM为新颖任务开发ML解决方案。我们展示了扩展LLM能力以理解结构化输入并针对解决新颖ML任务进行深入推理的可能性。研究发现，经过特定设计后，LLM能够（i）从现有ML任务经验中观察学习，（ii）通过有效推理为新任务生成有前景的结果。生成的解决方案可直接用于实现高度竞争力。