Language models often default to a narrow set of high-probability outputs, leaving their generation paths homogeneous and prone to mode collapse. Sampling-based strategies inject randomness but still struggle to guarantee diversity across multiple concurrent generation runs. We address this limitation by introducing STARS ($\textbf{St}$iefel-based $\textbf{A}$ctivation Steering for Diverse $\textbf{R}$ea$\textbf{S}$oning), a training-free, inference-time intervention method that transforms activation steering into an exploration engine. At each token, STARS collects the hidden activations of concurrent generation runs and optimizes multiple additive steering directions jointly on the Stiefel manifold. STARS maximizes the geometric volume of the steered activations, while the Stiefel manifold induces orthogonality of the steering interventions. This formulation explicitly promotes divergent activation vectors of concurrent generation runs, and implicitly promotes divergent generation trajectories. This manifold optimization formulation can be solved using a Riemannian gradient descent algorithm with convergence guarantees, but this algorithm is too time-consuming for real-time inference. To guarantee low latency, we further design a lightweight one-step update with an aggressive, closed-form stepsize. For test case generation and scientific discovery benchmarks, STARS consistently outperforms standard sampling methods, achieving greater diversity without sacrificing qualitative performance.

翻译：语言模型通常默认生成一组狭窄的高概率输出，导致其生成路径同质化且易陷入模式坍缩。基于采样的策略虽能注入随机性，但仍难以保证多个并行生成运行间的多样性。为克服这一局限，我们提出STARS（基于Stiefel的激活引导实现多样化推理），一种无需训练、在推理时进行干预的方法，将激活引导转化为探索引擎。在每个词元生成时，STARS收集并行生成运行的隐藏激活，并在Stiefel流形上联合优化多个附加引导方向。STARS最大化被引导激活的几何体积，而Stiefel流形则强制引导干预的正交性。该公式显式地促进并行生成运行间激活向量的发散，并隐式地促进生成轨迹的发散。此流形优化问题可采用具有收敛保证的黎曼梯度下降算法求解，但该算法在实时推理中耗时过多。为保证低延迟，我们进一步设计了一种轻量级单步更新方法，采用激进的闭式步长。在测试用例生成和科学发现基准测试中，STARS始终优于标准采样方法，在不牺牲质量性能的前提下实现了更高的多样性。