Methods: we adopted a biological networks approach that enables the systematic interrogation of ChatGPT's linked entities. In particular, we designed an ontology-driven fact-checking algorithm that compares biological graphs constructed from approximately 200,000 PubMed abstracts with counterparts constructed from a dataset generated using the ChatGPT-3.5 Turbo model. The nodes refer to biological entities (genes and diseases) that occur in the text. The edges represent the co-occurrence relationships of two entities mentioned in the same document, weighted by the proximity distance between these two entities. This research assumes a ``closed-world assumption'', meaning that fact-checking is performed only using the literature dataset as our ground truth. Results: in ten samples of 250 randomly selected records from the ChatGPT dataset of 1000 ``simulated'' articles , the fact-checking link accuracy ranged from 70% to 86%, while the remainder of the links remained unverified. Given the closed world assumption, the fact-checking precision is significant. When measuring and comparing the proximity distances of the edges of literature graphs against ChatGPT graphs we found that the ChatGPT distances were significantly shorter (ranging from 90 to 153) character distance. In contrast, the proximity distance of biological entities identified in the literature ranged from 236 to 765 character distance. This pattern held true for all the relationships among biological entities in the ten samples. Conclusion: this study demonstrated a reasonably high percentage accuracy of aggregate fact-checking of disease-gene relationships found in ChatGPT-generated texts. The strikingly consistent pattern of short proximity distances across all samples offers an illuminating feedback to the biological knowledge we possess in the literature today.

翻译：方法：我们采用生物网络方法，能够系统性核查ChatGPT的关联实体。具体而言，我们设计了一种基于本体驱动的事实核查算法，将基于约20万篇PubMed摘要构建的生物图谱与基于ChatGPT-3.5 Turbo模型生成数据集构建的对应图谱进行比较。节点指代文本中出现的生物实体（基因与疾病），边代表同一文献中提及的两种实体间的共现关系，权重则由这两种实体间的邻近距离决定。本研究采用"封闭世界假设"，即仅以文献数据集作为事实核查的基准真相。结果：在从包含1000篇"模拟"文章的ChatGPT数据集中随机抽取的250条记录的十组样本中，事实核查链接准确率在70%至86%之间，其余链接未被验证。在封闭世界假设下，事实核查精度具有显著意义。当测量并比较文献图谱与ChatGPT图谱的边邻近距离时，我们发现ChatGPT距离显著更短（范围在90至153个字符距离），而文献中识别的生物实体邻近距离范围为236至765个字符距离。这一模式在十组样本的所有生物实体关系中均保持成立。结论：本研究证明，对ChatGPT生成文本中疾病-基因关系进行汇总性事实核查，具有相当高的百分比准确率。所有样本中一致呈现的短邻近距离模式，为当前文献中我们掌握的生物学知识提供了富有启示性的反馈。