We introduce {\em generative monoculture}, a behavior observed in large language models (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for a given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g., LLMs more often produce efficient code), the dangers are exacerbated in others (e.g., LLMs refuse to share diverse opinions). As LLMs are increasingly used in high-impact settings such as education and web search, careful maintenance of LLM output diversity is essential to ensure a variety of facts and perspectives are preserved over time. We experimentally demonstrate the prevalence of generative monoculture through analysis of book review and code generation tasks, and find that simple countermeasures such as altering sampling or prompting strategies are insufficient to mitigate the behavior. Moreover, our results suggest that the root causes of generative monoculture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.

翻译：本文提出"生成性单一文化"这一概念，用以描述大语言模型（LLMs）在特定任务中表现出的行为特征：相较于可用训练数据的多样性，模型输出呈现显著收窄趋势。例如，对于评价褒贬不一的书籍，模型仅生成正面书评。虽然在某些场景下（如LLMs更倾向于生成高效代码），生成性单一文化可能提升性能，但在其他场景（如LLMs拒绝提供多元观点）中其危害性会被放大。随着LLMs在教育、网络搜索等高影响力领域日益普及，审慎维护模型输出多样性对于保障事实与观点的长期多元呈现至关重要。我们通过书评生成与代码生成任务的实验分析，证实了生成性单一文化的普遍性，并发现调整采样策略或提示工程等简单对策不足以缓解该现象。进一步研究表明，生成性单一文化的根源可能深植于LLMs的对齐过程，这提示我们需要开发能够保持或促进多样性的微调范式。