ChatGPT (GPT) has become one of the most talked-about innovations in recent years, with over 100 million users worldwide. However, there is still limited knowledge about the sources of information GPT utilizes. As a result, we carried out a study focusing on the sources of information within the field of environmental science. In our study, we asked GPT to identify the ten most significant subdisciplines within the field of environmental science. We then asked it to compose a scientific review article on each subdiscipline, including 25 references. We proceeded to analyze these references, focusing on factors such as the number of citations, publication date, and the journal in which the work was published. Our findings indicate that GPT tends to cite highly-cited publications in environmental science, with a median citation count of 1184.5. It also exhibits a preference for older publications, with a median publication year of 2010, and predominantly refers to well-respected journals in the field, with Nature being the most cited journal by GPT. Interestingly, our findings suggest that GPT seems to exclusively rely on citation count data from Google Scholar for the works it cites, rather than utilizing citation information from other scientific databases such as Web of Science or Scopus. In conclusion, our study suggests that Google Scholar citations play a significant role as a predictor for mentioning a study in GPT-generated content. This finding reinforces the dominance of Google Scholar among scientific databases and perpetuates the Matthew Effect in science, where the rich get richer in terms of citations. With many scholars already utilizing GPT for literature review purposes, we can anticipate further disparities and an expanding gap between lesser-cited and highly-cited publications.
翻译:ChatGPT(GPT)已成为近年来最受关注的创新之一,全球用户超过1亿。然而,关于GPT所利用的信息来源,我们仍知之甚少。为此,我们开展了一项聚焦环境科学领域信息来源的研究。研究中,我们让GPT识别环境科学领域十个最重要的分支学科,随后要求它针对每个分支学科撰写一篇包含25篇参考文献的科学综述文章。我们进一步分析了这些参考文献,重点关注引用次数、发表日期以及发表期刊等因素。结果表明,GPT倾向于引用环境科学领域高被引文献,中位被引次数达1184.5次;它也更偏好较早的文献,中位发表年份为2010年;并且主要引用该领域知名期刊,其中《自然》是被GPT引用最多的期刊。有趣的是,我们的发现表明,GPT似乎完全依赖谷歌学术的引用计数数据来确定所引用的文献,而非使用Web of Science或Scopus等其他科学数据库的引用信息。总之,本研究表明,谷歌学术引用计数是预测GPT生成内容中提及某项研究的重要指标。这一发现强化了谷歌学术在科学数据库中的主导地位,并延续了科学界的马太效应——即引用上“富者愈富”。鉴于已有众多学者利用GPT进行文献综述,可以预见,低被引与高被引出版物之间的差距将进一步扩大。