Large language models (LLMs) achieve strong performance in long-horizon decision-making tasks through multi-step interaction and reasoning at test time. While practitioners commonly believe a higher task success rate necessitates the use of a larger and stronger LLM model, multi-step interaction with a large LLM incurs prohibitive inference cost. To address this problem, we explore the use of low-precision quantized LLMs in the long-horizon decision-making process. Based on the observation of diverse sensitivities among interaction steps, we propose Dynamic Mixed-Precision Routing (DMR), a framework that adaptively selects between high-precision and low-precision LLMs at each decision step. The router is trained via a two-stage pipeline, consisting of KL-divergence-based supervised learning that identifies precision-sensitive steps, followed by Group-Relative Policy Optimization (GRPO) to further improve task success rates. Experiments on ALFWorld and WebShop demonstrate that our approach achieves a strong accuracy-cost trade-off over single-precision baselines.

翻译：大型语言模型（LLMs）通过多步交互和测试时推理，在长周期决策任务中展现出强大性能。尽管从业者普遍认为更高的任务成功率需要使用更大更强的LLM模型，但与大型LLM的多步交互会产生高昂的推理成本。为解决这一问题，我们探索了在长周期决策过程中使用低精度量化LLM的方法。基于交互步骤间敏感性差异的观察，我们提出动态混合精度路由（DMR）框架，在每步决策中自适应选择高精度或低精度LLM。路由器通过两阶段流水线训练：首先基于KL散度的监督学习识别精度敏感步骤，随后通过群组相对策略优化（GRPO）进一步提升任务成功率。在ALFWorld和WebShop上的实验表明，我们的方法相较于单精度基线实现了强准确率-成本权衡。