Large language models (LLMs) such as GPT-4 have exhibited remarkable performance in a variety of tasks, but this strong performance often comes with the high expense of using paid API services. In this paper, we are motivated to study building an LLM cascade to save the cost of using LLMs, particularly for performing reasoning (e.g., mathematical, causal) tasks. Our cascade pipeline follows the intuition that simpler questions can be addressed by a weaker but more affordable LLM, whereas only the challenging questions necessitate the stronger and more expensive LLM. To realize this decision-making, we consider the "answer consistency" of the weaker LLM as a signal of the question difficulty and propose several methods for the answer sampling and consistency checking, including one leveraging a mixture of two thought representations (i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and stronger LLMs, respectively, we demonstrate that our proposed LLM cascades can achieve performance comparable to using solely the stronger LLM but require only 40% of its cost.

翻译：大语言模型（如GPT-4）在各类任务中展现出卓越性能，但这种强大性能往往伴随着使用付费API服务的高昂成本。本文致力于研究构建大语言模型级联架构，以降低使用大语言模型执行推理任务（如数学推理、因果推理）时的成本。我们的级联流水线遵循以下直觉：简单问题可由性能较弱但成本更低的LLM处理，而只有具有挑战性的问题才需要调用更强且更昂贵的LLM。为实现这一决策机制，我们采用弱LLM的"答案一致性"作为问题难度的信号，并提出了多种答案采样与一致性检测方法，其中包括一种利用两种思维表征（思维链与程序思维）混合的方法。通过六个推理基准数据集的实验（以GPT-3.5-turbo和GPT-4分别作为弱/强LLM），我们证明所提出的LLM级联架构能达到与单独使用强LLM相当的性能，但成本仅为其40%。