Large language models (LLMs) have achieved remarkable progress in various natural language processing tasks with emergent abilities. However, they face inherent limitations, such as an inability to access up-to-date information, utilize external tools, or perform precise mathematical reasoning. In this paper, we introduce Chameleon, a plug-and-play compositional reasoning framework that augments LLMs to help address these challenges. Chameleon synthesizes programs to compose various tools, including LLM models, off-the-shelf vision models, web search engines, Python functions, and rule-based modules tailored to user interests. Built on top of an LLM as a natural language planner, Chameleon infers the appropriate sequence of tools to compose and execute in order to generate a final response. We showcase the adaptability and effectiveness of Chameleon on two tasks: ScienceQA and TabMWP. Notably, Chameleon with GPT-4 achieves an 86.54% accuracy on ScienceQA, significantly improving upon the best published few-shot model by 11.37%; using GPT-4 as the underlying LLM, Chameleon achieves a 17.8% increase over the state-of-the-art model, leading to a 98.78% overall accuracy on TabMWP. Further studies suggest that using GPT-4 as a planner exhibits more consistent and rational tool selection and is able to infer potential constraints given the instructions, compared to other LLMs like ChatGPT.

翻译：大型语言模型（LLM）凭借其涌现能力，在各类自然语言处理任务中取得了显著进展。然而，它们仍面临固有局限，例如无法获取最新信息、无法利用外部工具或执行精确的数学推理。本文提出Chameleon（变色龙），一种即插即用的组合推理框架，通过增强LLM以应对上述挑战。Chameleon综合生成程序来组合多种工具，包括LLM模型、现成的视觉模型、网络搜索引擎、Python函数以及根据用户兴趣定制的基于规则的模块。该框架以LLM作为自然语言规划器，推断并执行合适的工具序列以生成最终响应。我们通过ScienceQA和TabMWP两项任务展示了Chameleon的适应性与有效性。值得注意的是，基于GPT-4的Chameleon在ScienceQA上达到86.54%的准确率，较最佳少样本模型提升11.37%；当使用GPT-4作为底层LLM时，Chameleon在TabMWP上较现有最优模型提升17.8%，总体准确率达98.78%。进一步研究表明，与ChatGPT等其他LLM相比，采用GPT-4作为规划器时，框架能展现更一致且合理的工具选择能力，并可根据指令推断潜在约束。