Artificial Intelligence (AI) has made incredible progress recently. On the one hand, advanced foundation models like ChatGPT can offer powerful conversation, in-context learning and code generation abilities on a broad range of open-domain tasks. They can also generate high-level solution outlines for domain-specific tasks based on the common sense knowledge they have acquired. However, they still face difficulties with some specialized tasks because they lack enough domain-specific data during pre-training or they often have errors in their neural network computations on those tasks that need accurate executions. On the other hand, there are also many existing models and systems (symbolic-based or neural-based) that can do some domain-specific tasks very well. However, due to the different implementation or working mechanisms, they are not easily accessible or compatible with foundation models. Therefore, there is a clear and pressing need for a mechanism that can leverage foundation models to propose task solution outlines and then automatically match some of the sub-tasks in the outlines to the off-the-shelf models and systems with special functionalities to complete them. Inspired by this, we introduce TaskMatrix.AI as a new AI ecosystem that connects foundation models with millions of APIs for task completion. Unlike most previous work that aimed to improve a single AI model, TaskMatrix.AI focuses more on using existing foundation models (as a brain-like central system) and APIs of other AI models and systems (as sub-task solvers) to achieve diversified tasks in both digital and physical domains. As a position paper, we will present our vision of how to build such an ecosystem, explain each key component, and use study cases to illustrate both the feasibility of this vision and the main challenges we need to address next.

翻译：人工智能（AI）近期取得了令人瞩目的进展。一方面，像ChatGPT这样的先进基础模型能够针对广泛开放领域任务，提供强大的对话、上下文学习以及代码生成能力。它们还能基于所习得的常识知识，为特定领域任务生成高级解决方案大纲。然而，由于在预训练过程中缺乏足够的领域特定数据，或者针对需要精确执行的复杂任务时，神经网络计算常出现误差，这类模型在处理某些专业化任务时仍面临困难。另一方面，许多现有模型和系统（无论是基于符号还是基于神经网络的）能够出色地完成特定领域任务，但由于实现机制或工作方式不同，它们难以被基础模型便捷调用或与之兼容。因此，当前迫切需要一种机制，能够利用基础模型提出任务解决方案大纲，并自动将大纲中的部分子任务匹配到具有特定功能的现有模型和系统，从而完成整个任务。受此启发，我们提出TaskMatrix.AI——一种将基础模型与数百万个API连接起来以完成任务的AI生态系统。与以往大多数旨在改进单一AI模型的研究不同，TaskMatrix.AI更侧重于利用现有基础模型（作为类脑中央系统）以及其他AI模型和系统的API（作为子任务求解器），以完成数字和物理领域的多样化任务。作为一份立场论文，我们将阐述构建此类生态系统的愿景，解析每个关键组件，并通过案例研究说明该愿景的可行性以及下一步需要解决的主要挑战。