Large language models (LLMs) have led to a surge in collaborative writing with model assistance. As different users incorporate suggestions from the same model, there is a risk of decreased diversity in the produced content, potentially limiting diverse perspectives in public discourse. In this work, we measure the impact of co-writing on diversity via a controlled experiment, where users write argumentative essays in three setups -- using a base LLM (GPT3), a feedback-tuned LLM (InstructGPT), and writing without model help. We develop a set of diversity metrics and find that writing with InstructGPT (but not the GPT3) results in a statistically significant reduction in diversity. Specifically, it increases the similarity between the writings of different authors and reduces the overall lexical and content diversity. We additionally find that this effect is mainly attributable to InstructGPT contributing less diverse text to co-written essays. In contrast, the user-contributed text remains unaffected by model collaboration. This suggests that the recent improvement in generation quality from adapting models to human feedback might come at the cost of more homogeneous and less diverse content.

翻译：大语言模型（LLMs）的兴起引发了与模型辅助协作写作的浪潮。当不同用户采纳相同模型的建议时，所生成的内容存在多样性降低的风险，可能限制公共讨论中多元视角的表达。本研究通过控制实验测量协作写作对多样性的影响，让用户在三种场景下撰写议论文——使用基础LLM（GPT3）、使用经过反馈调优的LLM（InstructGPT）以及不使用模型辅助。我们开发了一套多样性指标，发现使用InstructGPT（而非GPT3）写作会导致多样性出现统计意义上的显著降低。具体而言，该模型不仅增加了不同作者文章间的相似度，还降低了整体词汇与内容多样性。我们进一步发现，这种效应主要归因于InstructGPT为协作论文贡献了更少样化的文本，而用户自主撰写的部分并未受到模型协作的影响。这表明，近期通过使模型适应人类反馈来提升生成质量的努力，可能以牺牲内容多样性和同质化加剧为代价。