Large language models (LLMs) such as GPT-4 have exhibited remarkable performance in a variety of tasks, but this strong performance often comes with the high expense of using paid API services. In this paper, we are motivated to study building an LLM cascade to save the cost of using LLMs, particularly for performing reasoning (e.g., mathematical, causal) tasks. Our cascade pipeline follows the intuition that simpler questions can be addressed by a weaker but more affordable LLM, whereas only the challenging questions necessitate the stronger and more expensive LLM. To realize this decision-making, we consider the "answer consistency" of the weaker LLM as a signal of the question difficulty and propose several methods for the answer sampling and consistency checking, including one leveraging a mixture of two thought representations (i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and stronger LLMs, respectively, we demonstrate that our proposed LLM cascades can achieve performance comparable to using solely the stronger LLM but require only 40% of its cost.

翻译：大语言模型（如GPT-4）在各种任务中展现出卓越性能，但这种强性能往往伴随着使用付费API服务的高昂成本。本文旨在研究构建大语言模型级联以降低使用大语言模型的成本，特别是针对推理任务（如数学推理、因果推理）。我们的级联流水线遵循以下直觉：简单问题可由能力较弱但成本更低的语言模型处理，而只有具有挑战性的问题才需要调用能力更强但成本更高的语言模型。为实现这种决策机制，我们将较弱模型的"答案一致性"视为问题难度的信号，并提出了多种答案采样与一致性检查方法，其中包括一种融合两种思维表示（即Chain-of-Thought与Program-of-Thought）的方法。通过在六个推理基准数据集上开展实验（以GPT-3.5-turbo和GPT-4分别作为较弱和较强模型），我们证明所提出的语言模型级联方法能够在仅消耗40%成本的情况下，达到与单独使用较强模型相当的性能。