When posed with prompts that permit a large number of valid answers, comprehensively generating them is the first step towards satisfying a wide range of users. In this paper, we study methods to elicit a comprehensive set of valid responses. To evaluate this, we introduce \textbf{diversity coverage}, a metric that measures the total quality scores assigned to each \textbf{unique} answer in the predicted answer set relative to the best possible answer set with the same number of answers. Using this metric, we evaluate 18 LLMs, finding no single model dominates at generating diverse responses to a wide range of open-ended prompts. Yet, per each prompt, there exists a model that outperforms all other models significantly at generating a diverse answer set. Motivated by this finding, we introduce a router that predicts the best model for each query. On NB-Wildchat, our trained router outperforms the single best model baseline (26.3% vs $23.8%). We further show generalization to an out-of-domain dataset (NB-Curated) as well as different answer-generation prompting strategies. Our work lays foundation for studying generating comprehensive answers when we have access to a suite of models.

翻译：面对允许大量有效答案的提示时，全面生成这些答案是满足广泛用户需求的首要步骤。本文研究了获取全面有效回答集的方法。为评估这一点，我们引入了**多样性覆盖度**这一指标，用于衡量预测答案集中每个**唯一**答案相较于相同数量答案的最优可能答案集所获得的总质量分数。通过该指标，我们评估了18个大语言模型，发现没有单一模型能在各类开放式提示的多样化响应生成中占据主导地位。然而对于每个提示而言，总存在某个模型在生成多样化答案集方面显著优于其他所有模型。基于这一发现，我们提出了一种路由学习器，能够预测每个查询对应的最优模型。在NB-Wildchat数据集上，我们训练的路由器性能优于单一最优模型基线（26.3% vs 23.8%）。我们进一步证明该方法可泛化至领域外数据集（NB-Curated）以及不同的答案生成提示策略。本研究为在模型集群环境下研究全面答案生成奠定了基础。