Recent breakthroughs in Large Reasoning Models (LRMs) have demonstrated that extensive Chain-of-Thought (CoT) generation is critical for enabling intricate cognitive behaviors, such as self-verification and backtracking, to solve complex tasks. However, this capability often leads to ``overthinking'', where models generate redundant reasoning paths that inflate computational costs without improving accuracy. While Supervised Fine-Tuning (SFT) on reasoning traces is a standard paradigm for the 'cold start' phase, applying existing compression techniques to these traces often compromises logical coherence or incurs prohibitive sampling costs. In this paper, we introduce ConMax (Confidence-Maximizing Compression), a novel reinforcement learning framework designed to automatically compress reasoning traces while preserving essential reasoning patterns. ConMax formulates compression as a reward-driven optimization problem, training a policy to prune redundancy by maximizing a weighted combination of answer confidence for predictive fidelity and thinking confidence for reasoning validity through a frozen auxiliary LRM. Extensive experiments across five reasoning datasets demonstrate that ConMax achieves a superior efficiency-performance trade-off. Specifically, it reduces inference length by 43% over strong baselines at the cost of a mere 0.7% dip in accuracy, proving its effectiveness in generating high-quality, efficient training data for LRMs.

翻译：大型推理模型（LRMs）的最新突破表明，广泛的思维链（CoT）生成对于实现复杂认知行为（如自我验证和回溯）以解决复杂任务至关重要。然而，这种能力常导致“过度思考”现象，即模型生成冗余的推理路径，在不提升准确性的情况下大幅增加计算成本。尽管在推理轨迹上进行监督微调（SFT）是“冷启动”阶段的标准范式，但将现有压缩技术应用于这些轨迹往往会损害逻辑连贯性或产生难以承受的采样成本。本文提出ConMax（置信度最大化压缩），一种新颖的强化学习框架，旨在自动压缩推理轨迹的同时保留核心推理模式。ConMax将压缩问题构建为奖励驱动的优化问题，通过训练一个策略来剪枝冗余内容，该策略通过冻结的辅助LRM最大化预测保真度的答案置信度与推理有效性的思考置信度的加权组合。在五个推理数据集上的大量实验表明，ConMax实现了更优的效率-性能权衡。具体而言，其在仅损失0.7%准确度的代价下，将推理长度较强大基线降低了43%，证明了其为LRMs生成高质量高效训练数据的有效性。