We use prompt engineering to guide ChatGPT in the automation of text mining of metal-organic frameworks (MOFs) synthesis conditions from diverse formats and styles of the scientific literature. This effectively mitigates ChatGPT's tendency to hallucinate information -- an issue that previously made the use of Large Language Models (LLMs) in scientific fields challenging. Our approach involves the development of a workflow implementing three different processes for text mining, programmed by ChatGPT itself. All of them enable parsing, searching, filtering, classification, summarization, and data unification with different tradeoffs between labor, speed, and accuracy. We deploy this system to extract 26,257 distinct synthesis parameters pertaining to approximately 800 MOFs sourced from peer-reviewed research articles. This process incorporates our ChemPrompt Engineering strategy to instruct ChatGPT in text mining, resulting in impressive precision, recall, and F1 scores of 90-99%. Furthermore, with the dataset built by text mining, we constructed a machine-learning model with over 86% accuracy in predicting MOF experimental crystallization outcomes and preliminarily identifying important factors in MOF crystallization. We also developed a reliable data-grounded MOF chatbot to answer questions on chemical reactions and synthesis procedures. Given that the process of using ChatGPT reliably mines and tabulates diverse MOF synthesis information in a unified format, while using only narrative language requiring no coding expertise, we anticipate that our ChatGPT Chemistry Assistant will be very useful across various other chemistry sub-disciplines.
翻译:我们利用提示工程(prompt engineering)引导ChatGPT,从科学文献中不同格式和风格的文本中自动挖掘金属有机框架(MOFs)的合成条件。这有效缓解了ChatGPT编造信息的倾向——该问题此前使得大型语言模型(LLMs)在科学领域的应用充满挑战。我们的方法涉及开发一个工作流程,其中包含由ChatGPT自身编程实现的三种不同文本挖掘过程。这些过程均支持解析、搜索、过滤、分类、总结和数据统一,并在人工投入、速度和准确性之间实现不同权衡。我们部署该系统从同行评议研究文章中提取了约800种MOF的26,257个不同合成参数。该过程整合了我们的化学提示工程策略以指导ChatGPT进行文本挖掘,实现了90-99%的惊人精确率、召回率与F1分数。此外,利用文本挖掘构建的数据集,我们建立了一个机器学习模型,预测MOF实验结晶结果的准确率超过86%,并初步识别了MOF结晶中的重要因素。我们还开发了一个基于可靠数据驱动的MOF聊天机器人,用于回答化学反应和合成程序相关的问题。鉴于该过程能够以统一格式可靠地挖掘和整理多种MOF合成信息,且仅需自然语言描述而无需编程技能,我们预计ChatGPT化学助手将在化学其他多个子学科中发挥重要作用。