Large language models (LLMs) have shown impressive capabilities in various tasks, yet they still struggle with math reasoning. Despite efforts to optimize Chain-of-Thoughts (CoT) prompts and fine-tune LLMs, the potential of few-shot learning remains unexplored. In this work, we propose CoT-Influx, a novel approach pushing the boundaries of few-shot CoT learning to improve LLM math reasoning capabilities. CoT-Influx addresses the challenges of the selection of useful examples and limited number of examples due to restricted context window length. Inspired by our observation that natural language inputs contain many redundancy, we propose a coarse-to-fine pruner as a plug-and-play module for LLMs, which first identifies as many crucial CoT examples as possible and then further prunes unimportant tokens within the context window. To train the pruner, we collect a math reasoning dataset with diverse difficulty and steps, introduce a reward to measure both the input's effectiveness for math reasoning and token length constraints, and propose a novel training approach with reinforcement learning. As a result, CoT-Influx significantly outperforms CoT and few-shot prompting baselines across various LLMs (LLaMA2-7B, 13B, 70B) and 5 mathematical datasets, achieving up to 4.55% absolute improvements. Remarkably, without any fine-tuning, LLaMA2-70B with CoT-Influx surpasses GPT-3.5 and a wide range of larger LLMs (PaLM, Minerva, etc.) on the GSM8K.

翻译：大型语言模型（LLM）在各类任务中展现出强大能力，但在数学推理方面仍然存在困难。尽管已有研究致力于优化思维链（CoT）提示和微调LLM，但少样本学习的潜力尚未被充分发掘。本文提出CoT-Influx这一创新方法，旨在突破少样本思维链学习的边界，以提升LLM的数学推理能力。CoT-Influx解决了有效示例选择以及因上下文窗口长度限制导致的示例数量不足两大难题。受自然语言输入存在大量冗余这一现象的启发，我们提出一种可插拔的粗到细剪枝器，该模块首先尽可能多地筛选出关键CoT示例，然后进一步剪枝上下文窗口中的非必要标记。为了训练该剪枝器，我们收集了一个包含不同难度和推理步骤的数学推理数据集，引入了一个同时衡量输入对数学推理的有效性及标记长度约束的奖励函数，并提出了一种基于强化学习的新型训练方法。实验结果表明，CoT-Influx在多种LLM（LLaMA2-7B、13B、70B）及5个数学数据集上显著优于CoT和少样本提示基线方法，绝对性能提升最高达4.55%。值得注意的是，无需任何微调，配备CoT-Influx的LLaMA2-70B在GSM8K数据集上便超越了GPT-3.5及众多更大规模的LLM（如PaLM、Minerva等）。