Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are numerous AI models available for various domains and modalities, they cannot handle complicated AI tasks autonomously. Considering large language models (LLMs) have exhibited exceptional abilities in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks, with language serving as a generic interface to empower this. Based on this philosophy, we present HuggingGPT, an LLM-powered agent that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., Hugging Face) to solve AI tasks. Specifically, we use ChatGPT to conduct task planning when receiving a user request, select models according to their function descriptions available in Hugging Face, execute each subtask with the selected AI model, and summarize the response according to the execution results. By leveraging the strong language capability of ChatGPT and abundant AI models in Hugging Face, HuggingGPT can tackle a wide range of sophisticated AI tasks spanning different modalities and domains and achieve impressive results in language, vision, speech, and other challenging tasks, which paves a new way towards the realization of artificial general intelligence.

翻译：解决跨不同领域和模态的复杂AI任务是实现通用人工智能的关键一步。尽管存在大量面向不同领域和模态的AI模型，但它们在自主处理复杂AI任务方面仍有不足。鉴于大型语言模型在语言理解、生成、交互和推理方面展现出卓越能力，我们主张LLMs可作为控制器来管理现有AI模型，通过将语言作为通用接口赋能复杂AI任务的求解。基于这一理念，我们提出HuggingGPT——一种由LLM驱动的智能体，它利用LLMs（如ChatGPT）连接机器学习社区（如Hugging Face）中的各类AI模型来解决AI任务。具体而言，我们在接收用户请求后使用ChatGPT进行任务规划，依据Hugging Face中模型的功能描述进行模型选择，利用所选AI模型执行各子任务，并基于执行结果生成总结响应。通过融合ChatGPT强大的语言能力与Hugging Face丰富的AI模型资源，HuggingGPT能够处理横跨不同模态和领域的多样化复杂AI任务，在语言、视觉、语音及其他具有挑战性的任务中取得显著成效，这为实现通用人工智能开辟了新路径。