There has been a steep recent increase in the number of large language model (LLM) papers, producing a dramatic shift in the scientific landscape which remains largely undocumented through bibliometric analysis. Here, we analyze 388K papers posted on the CS and Stat arXivs, focusing on changes in publication patterns in 2023 vs. 2018-2022. We analyze how the proportion of LLM papers is increasing; the LLM-related topics receiving the most attention; the authors writing LLM papers; how authors' research topics correlate with their backgrounds; the factors distinguishing highly cited LLM papers; and the patterns of international collaboration. We show that LLM research increasingly focuses on societal impacts: there has been an 18x increase in the proportion of LLM-related papers on the Computers and Society sub-arXiv, and authors newly publishing on LLMs are more likely to focus on applications and societal impacts than more experienced authors. LLM research is also shaped by social dynamics: we document gender and academic/industry disparities in the topics LLM authors focus on, and a US/China schism in the collaboration network. Overall, our analysis documents the profound ways in which LLM research both shapes and is shaped by society, attesting to the necessity of sociotechnical lenses.

翻译：近年来，大型语言模型（LLM）论文数量急剧增加，引发了科学领域的显著变革，但这一变革尚未通过文献计量分析得到充分记录。本文分析了CS和Stat arXiv上发布的38.8万篇论文，重点关注2023年相比2018-2022年发表模式的变化。我们考察了LLM论文比例的增长趋势；最受关注的LLM相关主题；撰写LLM论文的作者群体；作者研究主题与其背景的相关性；区分高被引LLM论文的关键因素；以及国际合作模式。研究表明，LLM研究日益聚焦社会影响：计算机与社会子arXiv上LLM相关论文的比例增长了18倍，且新发表LLM论文的作者比资深作者更关注应用与社会影响。LLM研究亦受社会动态塑造：我们记录了LLM作者研究主题中的性别与学术/产业差异，以及合作网络中的中美分裂现象。总体而言，我们的分析揭示了LLM研究以深刻方式双向塑造社会与被社会塑造的过程，证实了社会技术视角的必要性。