Large Language Models (LLMs) are rapidly reshaping scientific research. We analyze these changes in multiple, large-scale datasets with 2.1M preprints, 28K peer review reports, and 246M online accesses to scientific documents. We find: 1) scientists adopting LLMs to draft manuscripts demonstrate a large increase in paper production, ranging from 23.7-89.3% depending on scientific field and author background, 2) LLM use has reversed the relationship between writing complexity and paper quality, leading to an influx of manuscripts that are linguistically complex but substantively underwhelming, and 3) LLM adopters access and cite more diverse prior work, including books and younger, less-cited documents. These findings highlight a stunning shift in scientific production that will likely require a change in how journals, funding agencies, and tenure committees evaluate scientific works.
翻译:大型语言模型(LLMs)正在迅速重塑科学研究。我们通过分析多个大规模数据集(包含210万份预印本、2.8万份同行评审报告以及2.46亿次科学文献在线访问)来解析这些变化。研究发现:1)采用LLMs起草文稿的科学家的论文产出量大幅增长,增幅在23.7%至89.3%之间,具体取决于科学领域和作者背景;2)LLMs的使用逆转了写作复杂度与论文质量之间的关系,导致大量语言复杂但实质内容乏善可陈的稿件涌入;3)LLMs采用者会查阅并引用更多样化的先前工作,包括书籍以及发表时间更近、被引频次较低的文献。这些发现凸显了科学产出的惊人转变,这可能要求期刊、资助机构及终身教职评审委员会调整对科学成果的评估方式。