Reinforcement Learning with Verifiable Rewards (RLVR) is a central technique for improving long-horizon reasoning in Large Language Models (LLMs). However, existing RLVR methods often encourage unnecessarily long reasoning rollouts, which can degrade reasoning coherence and exhaust the available context budget. Existing approaches to long-context organization often depend on external mechanisms to organize rollouts, rather than enabling the model to manage its own reasoning trajectory. To address this limitation, we propose ReSum, a novel RLVR framework that enables LLMs to compress and organize their reasoning trajectories through self-summarization. Our pilot studies show that self-summarization stabilizes generation by lowering token-level entropy, and that introducing a ``summarization'' phrase can substantially mitigate errors propagated from an incorrect rollout prefix. Motivated by these findings, ReSum adopts a summarization-aware adaptive rollout mechanism that contrastively evaluates whether self-summarization benefits the ongoing reasoning process. Specifically, when the model spontaneously triggers self-summarization, ReSum masks the summarization phrase to create a contrastive branch; for non-summarization positions, it instead randomly injects the phrase to create a matched branch. We further design a summarization-aware advantage to enable finer-grained comparison between contrastive rollout trajectories. Extensive experiments show that ReSum improves performance at an average of 4\% while reducing rollout length by 18.6\%.

翻译：基于可验证奖励的强化学习（RLVR）是提升大语言模型（LLM）长程推理能力的核心技术。然而现有RLVR方法常导致过度冗长的推理展开，这既削弱推理连贯性，又消耗有限的上下文预算。当前长上下文组织方法往往依赖外部机制整理展开内容，而非让模型自主管理推理轨迹。针对这一局限，我们提出ReSum——一种通过自摘要压缩与组织推理轨迹的新型RLVR框架。先导实验表明，自摘要能通过降低词元级熵值稳定生成过程，而引入"摘要"短语可显著抑制错误推理前缀的传播效应。基于此发现，ReSum采用摘要感知的自适应展开机制，通过对比评估判断自摘要是否有利于当前推理过程：当模型自发触发自摘要时，ReSum遮蔽摘要短语构建对照分支；在非摘要位置则随机注入该短语生成匹配分支。我们进一步设计摘要感知优势函数，以实现对比展开轨迹的细粒度比较。大量实验表明，ReSum在平均提升4%性能的同时将展开长度缩减18.6%。