Large Language Models (LLMs) for complex reasoning is often hindered by high computational costs and latency, while resource-efficient Small Language Models (SLMs) typically lack the necessary reasoning capacity. Existing collaborative approaches, such as cascading or routing, operate at a coarse granularity by offloading entire queries to LLMs, resulting in significant computational waste when the SLM is capable of handling the majority of reasoning steps. To address this, we propose RelayLLM, a novel framework for efficient reasoning via token-level collaborative decoding. Unlike routers, RelayLLM empowers the SLM to act as an active controller that dynamically invokes the LLM only for critical tokens via a special command, effectively "relaying" the generation process. We introduce a two-stage training framework, including warm-up and Group Relative Policy Optimization (GRPO) to teach the model to balance independence with strategic help-seeking. Empirical results across six benchmarks demonstrate that RelayLLM achieves an average accuracy of 49.52%, effectively bridging the performance gap between the two models. Notably, this is achieved by invoking the LLM for only 1.07% of the total generated tokens, offering a 98.2% cost reduction compared to performance-matched random routers.

翻译：大型语言模型（LLMs）在复杂推理任务中常受限于高昂的计算成本与延迟，而资源高效的小型语言模型（SLMs）通常缺乏必要的推理能力。现有的协作方法（如级联或路由）以粗粒度方式运行，将整个查询卸载给LLMs，当SLM能够处理大部分推理步骤时，会造成显著的计算资源浪费。为解决这一问题，我们提出了RelayLLM——一种基于词元级协作解码的新型高效推理框架。与路由机制不同，RelayLLM使SLM能够作为主动控制器，通过特殊指令动态地仅为关键词元调用LLM，从而有效地实现生成过程的“接力”。我们引入了一个两阶段训练框架，包括预热阶段和组相对策略优化（GRPO），以指导模型在自主推理与策略性求助之间取得平衡。在六个基准测试上的实证结果表明，RelayLLM取得了平均49.52%的准确率，有效弥合了两种模型之间的性能差距。值得注意的是，这一成果仅需为总生成词元的1.07%调用LLM即可实现，与性能匹配的随机路由方法相比，计算成本降低了98.2%。