Deployed large reasoning models (LRMs) often behave unexpectedly. Test-time steering controls LRM outputs by intervening on their hidden representations, but it can degrade output quality. We argue that prior steering work implicitly relies on internal features that detect behavior in already generated text. We show that these detection features are poor predictors of future behavioral outcomes, and thus not the natural intervention target. Instead, we train activation probes to predict future behavior likelihoods from intermediate reasoning steps. These probes predict the most likely behavior with 64%-91% accuracy, revealing a separate type of internal prediction features. Building on these prediction features, we introduce a text-level steering method, Future Probe Controlled Generation. FPCG samples multiple candidate sentences and chooses the best one according to a probe predicting the future behavior likelihood. This enables steering with almost no output quality degradation. FPCG also enables steering in several evaluations where activation steering fails. These results show that distinguishing detection and prediction features enables a more nuanced approach to controlling LRM behaviors.

翻译：部署的大型推理模型（LRMs）常出现意外行为。测试时引导通过干预模型隐藏表征来控制LRM输出，但可能降低输出质量。我们认为，此前引导工作隐含依赖于检测已生成文本行为的内部特征。研究表明，这些检测特征对未来行为结果的预测能力较差，因此并非自然干预目标。我们转而训练激活探针，从中间推理步骤预测未来行为概率。这些探针预测最可能行为的准确率达64%-91%，揭示了一种独立类型的内部预测特征。基于这些预测特征，我们提出文本级引导方法——未来探针控制生成（FPCG）。FPCG通过采样多个候选句子，并根据预测未来行为概率的探针选择最优输出，从而在几乎不损害输出质量的前提下实现引导。FPCG还在多项激活引导失效的评估中成功实现引导。这些结果表明，区分检测特征与预测特征能实现更精细化的LRM行为控制方法。