There has been a growing effort to replace manual extraction of data from research papers with automated data extraction based on natural language processing, language models, and recently, large language models (LLMs). Although these methods enable efficient extraction of data from large sets of research papers, they require a significant amount of up-front effort, expertise, and coding. In this work we propose the ChatExtract method that can fully automate very accurate data extraction with minimal initial effort and background, using an advanced conversational LLM. ChatExtract consists of a set of engineered prompts applied to a conversational LLM that both identify sentences with data, extract that data, and assure the data's correctness through a series of follow-up questions. These follow-up questions largely overcome known issues with LLMs providing factually inaccurate responses. ChatExtract can be applied with any conversational LLMs and yields very high quality data extraction. In tests on materials data we find precision and recall both close to 90% from the best conversational LLMs, like ChatGPT-4. We demonstrate that the exceptional performance is enabled by the information retention in a conversational model combined with purposeful redundancy and introducing uncertainty through follow-up prompts. These results suggest that approaches similar to ChatExtract, due to their simplicity, transferability, and accuracy are likely to become powerful tools for data extraction in the near future. Finally, databases for critical cooling rates of metallic glasses and yield strengths of high entropy alloys are developed using ChatExtract.

翻译：近年来，基于自然语言处理、语言模型及近期大规模语言模型（LLMs）的自动化数据提取方法，正逐步取代传统的人工从研究论文中提取数据。尽管这些技术能高效地从大量研究论文中提取数据，但需要大量的前期投入、专业知识背景及编程工作。本文提出ChatExtract方法，该方法利用先进的对话式LLM，能够以极低的前期准备工作和背景知识实现全自动、高精度的数据提取。ChatExtract通过一组针对对话式LLM精心设计的提示，识别包含数据的句子、提取数据，并通过一系列后续提问验证数据正确性。这些后续提问极大地克服了LLM生成不准确信息的已知问题。ChatExtract可适用于任何对话式LLM，并能获得高质量的数据提取结果。在材料数据测试中，使用最佳对话式LLM（如ChatGPT-4）时，其精确率与召回率均接近90%。研究表明，这种卓越性能得益于对话模型的信息保留能力，结合有意的冗余设计及通过后续提示引入的不确定性验证。这些结果表明，类似ChatExtract的方法因其简洁性、可迁移性和准确性，有望在不久的将来成为强大的数据提取工具。最后，本文利用ChatExtract构建了金属玻璃临界冷却速率与高熵合金屈服强度数据库。