Alignment has greatly improved large language models (LLMs)' output quality at the cost of diversity, yielding highly similar outputs across generations, especially in open-ended generation tasks. We propose Base-Aligned Model Collaboration (BACo), an inference-time token-level model collaboration framework that dynamically combines a base LLM with its aligned counterpart to optimize diversity and quality. Using uncertainty and content-based signals, BACo employs routing strategies to determine, at each token, which model to decode from. Prior diversity-promoting methods often improve diversity at the expense of quality or require expensive decoding or post-training. In contrast, BACo achieves both high diversity and quality post hoc within a single pass, while offering strong controllability. We introduce a family of effective routing strategies and evaluate them across three open-ended generation tasks with 13 diversity and quality metrics. BACo consistently surpasses state-of-the-art inference-time baselines. With our best router, BACo achieves a 21.3% joint improvement in diversity and quality, which is further supported by human evaluations. Overall, our results demonstrate that collaboration between base and aligned models provides an effective and controllable mechanism for optimizing the diversity-quality trade-off.

翻译：对齐技术大幅提升了大型语言模型（LLMs）的输出质量，但牺牲了多样性，导致各代模型在开放式生成任务中产生高度相似的结果。我们提出基座对齐模型协作（BACo）框架——一种推理阶段的词元级模型协作方法，通过动态结合基座LLM及其对齐版本以优化多样性与质量。BACo利用基于不确定性和内容的信号，采用路由策略在解码每个词元时确定由哪个模型生成。以往的多样性提升方法常以质量下降为代价，或需要昂贵的解码流程与后训练。相比之下，BACo在单次推理中即可事后实现高多样性与高质量兼具的效果，并具备强可控性。我们提出一系列高效路由策略，在三个开放式生成任务中通过13项多样性及质量指标进行评估。BACo始终超越现有最先进的推理阶段基线方法。采用最优路由策略时，BACo在多样性与质量上实现21.3%的联合提升，该结果进一步得到人工评估的验证。总体而言，我们的实验表明：基座模型与对齐模型间的协作为优化多样性与质量的权衡提供了有效且可控的机制。