Recent advances in Large Language Models (LLMs) have improved multi-step reasoning. Most approaches rely on Chain-of-Thought (CoT) rationales. Previous studies have shown that LLMs often generate logically inconsistent reasoning steps even when their final answers are correct. These inconsistencies reduce the reliability of step-level reasoning. We propose GeoSteer, a manifold-based framework that improves the quality of intermediate reasoning. The method consists of: (1) constructing a CoT dataset with segment-level scores, (2) training a Variational Autoencoder (VAE) model and a quality estimation model to learn a low-dimensional manifold of high-quality CoT trajectories, and (3) steering hidden states of target LLMs toward higher-quality regions in the latent space. This update in a latent space behaves like a natural-gradient adjustment in the original hidden-state space. It ensures geometrically coherent steering. We evaluate GeoSteer on the GSM8k dataset using the Qwen3 series. We measure via answer accuracy and overall reasoning performance. GeoSteer improved the exact match accuracy by up to 2.6 points. It also enhanced the pairwise win rate by 5.3 points. These results indicate that GeoSteer provides an effective and controllable mechanism for improving the quality of intermediate reasoning in LLMs.

翻译：大型语言模型（LLM）的最新进展提升了多步推理能力。现有方法大多依赖于思维链（CoT）推理过程。先前研究表明，即使最终答案正确，LLM生成的推理步骤仍常存在逻辑不一致问题，这种不一致性降低了步骤级推理的可靠性。本文提出GeoSteer——一种基于流形的框架，用于提升中间推理质量。该方法包含三个步骤：（1）构建具有分段评分的CoT数据集；（2）训练变分自编码器（VAE）模型与质量评估模型，以学习高质量CoT轨迹的低维流形；（3）将目标LLM的隐藏状态向潜在空间中更高质量区域引导。这种潜在空间中的更新类似于原始隐藏状态空间中的自然梯度调整，确保了几何连贯的引导效果。我们在GSM8k数据集上使用Qwen3系列模型评估GeoSteer，通过答案准确率和整体推理性能进行度量。实验表明：GeoSteer将精确匹配准确率最高提升2.6个百分点，配对胜率提升5.3个百分点。这些结果证明GeoSteer为提升LLM中间推理质量提供了有效且可控的机制。