Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence (AGI). While there are abundant AI models available for different domains and modalities, they cannot handle complicated AI tasks. Considering large language models (LLMs) have exhibited exceptional ability in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks and language could be a generic interface to empower this. Based on this philosophy, we present HuggingGPT, a system that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., HuggingFace) to solve AI tasks. Specifically, we use ChatGPT to conduct task planning when receiving a user request, select models according to their function descriptions available in HuggingFace, execute each subtask with the selected AI model, and summarize the response according to the execution results. By leveraging the strong language capability of ChatGPT and abundant AI models in HuggingFace, HuggingGPT is able to cover numerous sophisticated AI tasks in different modalities and domains and achieve impressive results in language, vision, speech, and other challenging tasks, which paves a new way towards AGI.

翻译：解决涉及不同领域与模态的复杂人工智能任务是迈向通用人工智能（AGI）的关键一步。尽管针对不同领域和模态已有大量可用的人工智能模型，但它们仍无法处理复杂的AI任务。考虑到大语言模型（LLMs）在语言理解、生成、交互和推理方面展现出卓越能力，我们主张将大语言模型作为控制器来管理现有AI模型以解决复杂AI任务，并认为语言可作为实现这一目标的通用接口。基于这一理念，我们提出HuggingGPT系统，该系统利用大语言模型（如ChatGPT）连接机器学习社区（如HuggingFace）中的各类AI模型以解决AI任务。具体而言，当收到用户请求时，我们使用ChatGPT进行任务规划，根据HuggingFace中提供的功能描述选择模型，使用所选AI模型执行每个子任务，并根据执行结果总结响应。通过利用ChatGPT强大的语言能力和HuggingFace丰富的AI模型资源，HuggingGPT能够覆盖不同模态和领域的众多复杂AI任务，并在语言、视觉、语音及其他具有挑战性的任务中取得显著成果，这为迈向AGI开辟了新路径。