Automated machine learning (AutoML) is a collection of techniques designed to automate the machine learning development process. While traditional AutoML approaches have been successfully applied in several critical steps of model development (e.g. hyperparameter optimization), there lacks a AutoML system that automates the entire end-to-end model production workflow. To fill this blank, we present AutoMMLab, a general-purpose LLM-empowered AutoML system that follows user's language instructions to automate the whole model production workflow for computer vision tasks. The proposed AutoMMLab system effectively employs LLMs as the bridge to connect AutoML and OpenMMLab community, empowering non-expert individuals to easily build task-specific models via a user-friendly language interface. Specifically, we propose RU-LLaMA to understand users' request and schedule the whole pipeline, and propose a novel LLM-based hyperparameter optimizer called HPO-LLaMA to effectively search for the optimal hyperparameters. Experiments show that our AutoMMLab system is versatile and covers a wide range of mainstream tasks, including classification, detection, segmentation and keypoint estimation. We further develop a new benchmark, called LAMP, for studying key components in the end-to-end prompt-based model training pipeline. Code, model, and data will be released.

翻译：自动机器学习（AutoML）是一套旨在自动化机器学习开发流程的技术体系。尽管传统AutoML方法已成功应用于模型开发中的多个关键环节（如超参数优化），但目前仍缺乏能够自动化整个端到端模型生产工作流的AutoML系统。为填补这一空白，我们提出AutoMMLab——一个通用型、由大型语言模型（LLM）驱动的AutoML系统，能够根据用户语言指令自动化完成计算机视觉任务的完整模型生产流程。所提出的AutoMMLab系统有效利用LLM作为桥梁连接AutoML与OpenMMLab社区，使非专业人士能通过友好的语言界面轻松构建特定任务模型。具体而言，我们提出RU-LLaMA理解用户请求并调度整个流水线，同时提出一种基于LLM的新型超参数优化器HPO-LLaMA，以高效搜索最优超参数。实验表明，AutoMMLab系统具有通用性，覆盖分类、检测、分割及关键点估计等主流任务。我们进一步开发了新基准LAMP，用于研究端到端提示式模型训练流水线中的关键组件。代码、模型及数据将公开发布。