Language Models (LMs) have proven to be useful in various downstream applications, such as summarisation, translation, question answering and text classification. LMs are becoming increasingly important tools in Artificial Intelligence, because of the vast quantity of information they can store. In this work, we present ProP (Prompting as Probing), which utilizes GPT-3, a large Language Model originally proposed by OpenAI in 2020, to perform the task of Knowledge Base Construction (KBC). ProP implements a multi-step approach that combines a variety of prompting techniques to achieve this. Our results show that manual prompt curation is essential, that the LM must be encouraged to give answer sets of variable lengths, in particular including empty answer sets, that true/false questions are a useful device to increase precision on suggestions generated by the LM, that the size of the LM is a crucial factor, and that a dictionary of entity aliases improves the LM score. Our evaluation study indicates that these proposed techniques can substantially enhance the quality of the final predictions: ProP won track 2 of the LM-KBC competition, outperforming the baseline by 36.4 percentage points. Our implementation is available on https://github.com/HEmile/iswc-challenge.

翻译：语言模型（LMs）已被证明在多种下游应用中具有实用性，例如摘要生成、翻译、问答和文本分类。由于能够存储海量信息，语言模型正成为人工智能领域日益重要的工具。在本研究中，我们提出ProP（提示即探测）方法，利用GPT-3（OpenAI于2020年提出的大型语言模型）执行知识库构建任务。ProP采用多步骤方法，结合多种提示技术来实现这一目标。研究结果表明：人工提示设计至关重要；必须鼓励语言模型生成可变长度的答案集，特别是包含空答案集；真/假问题是提升模型生成建议精度的有效手段；语言模型规模是关键因素；实体别名词典可提升模型得分。我们的评估研究表明，所提出的技术能显著提升最终预测质量：ProP在LM-KBC竞赛第二赛道中获胜，比基线高出36.4个百分点。我们的实现代码已开源在https://github.com/HEmile/iswc-challenge。