The advent of Large Language Models (LLM) has revolutionized the field of natural language processing, enabling significant progress in various applications. One key area of interest is the construction of Knowledge Bases (KB) using these powerful models. Knowledge bases serve as repositories of structured information, facilitating information retrieval and inference tasks. Our paper proposes LLM2KB, a system for constructing knowledge bases using large language models, with a focus on the Llama 2 architecture and the Wikipedia dataset. We perform parameter efficient instruction tuning for Llama-2-13b-chat and StableBeluga-13B by training small injection models that have only 0.05 % of the parameters of the base models using the Low Rank Adaptation (LoRA) technique. These injection models have been trained with prompts that are engineered to utilize Wikipedia page contexts of subject entities fetched using a Dense Passage Retrieval (DPR) algorithm, to answer relevant object entities for a given subject entity and relation. Our best performing model achieved an average F1 score of 0.6185 across 21 relations in the LM-KBC challenge held at the ISWC 2023 conference.

翻译：大语言模型（LLM）的出现彻底改变了自然语言处理领域，推动了各类应用的显著进展。其中，利用这些强大模型构建知识库（KB）成为关键研究方向。知识库作为结构化信息的存储库，有助于信息检索和推理任务。本文提出LLM2KB系统，该系统基于大语言模型构建知识库，重点采用Llama 2架构和维基百科数据集。我们通过训练小型注入模型（仅含基础模型参数的0.05%），结合低秩自适应（LoRA）技术，对Llama-2-13b-chat和StableBeluga-13B进行参数高效的指令调优。这些注入模型采用经过精心设计的提示词进行训练，利用稠密段落检索（DPR）算法获取主题实体的维基百科页面上下文，从而针对给定主题实体和关系生成对应的目标实体。在ISWC 2023会议举办的LM-KBC挑战赛中，我们表现最佳的模型在21个关系上的平均F1分数达到0.6185。