While large language models (LLMs) are generally considered proficient in generating language, how similar their language usage is to that of humans remains understudied. In this paper, we test whether models exhibit linguistic convergence, a core pragmatic element of human language communication: do models adapt, or converge, to the linguistic patterns of their user? To answer this, we systematically compare model completions of existing dialogues to original human responses across sixteen language models, three dialogue corpora, and various stylometric features. We find that models strongly converge to the conversation's style, often significantly overfitting relative to the human baseline. While convergence patterns are often feature-specific, we observe consistent shifts in convergence across modeling settings, with instruction-tuned and larger models converging less than their pretrained and smaller counterparts. Given the differences in human and model convergence patterns, we hypothesize that the underlying mechanisms driving these behaviors are very different.
翻译:尽管大型语言模型(LLM)通常被认为擅长生成语言,但其语言使用与人类的相似程度仍未得到充分研究。本文旨在检验模型是否表现出语言趋同性——人类语言交际的核心语用特征:模型是否会适应用户的语言模式或与之趋同?为此,我们系统性地比较了十六种语言模型在三个对话语料库上生成的对话补全与原始人类回应,并分析了多种文体计量特征。研究发现,模型会强烈趋同于对话的文体风格,其趋同程度常显著超过人类基线水平。虽然趋同模式常因特征而异,但我们观察到不同建模设置下趋同性存在系统性差异:经过指令微调的模型和较大规模模型的趋同性弱于预训练模型和较小规模模型。鉴于人类与模型的趋同模式存在差异,我们推测驱动这些行为的内在机制存在本质区别。