Large Language Models (LLMs) are capable of performing zero-shot closed-book question answering tasks, based on their internal knowledge stored in parameters during pre-training. However, such internalized knowledge might be insufficient and incorrect, which could lead LLMs to generate factually wrong answers. Furthermore, fine-tuning LLMs to update their knowledge is expensive. To this end, we propose to augment the knowledge directly in the input of LLMs. Specifically, we first retrieve the relevant facts to the input question from the knowledge graph based on semantic similarities between the question and its associated facts. After that, we prepend the retrieved facts to the input question in the form of the prompt, which is then forwarded to LLMs to generate the answer. Our framework, Knowledge-Augmented language model PromptING (KAPING), requires no model training, thus completely zero-shot. We validate the performance of our KAPING framework on the knowledge graph question answering task, that aims to answer the user's question based on facts over a knowledge graph, on which ours outperforms relevant zero-shot baselines by up to 48% in average, across multiple LLMs of various sizes.

翻译：大语言模型（LLMs）能够基于预训练过程中存储在参数中的内部知识执行零样本闭卷问答任务。然而，这些内部知识可能不充分或不准确，从而导致LLMs生成事实错误的答案。此外，对LLMs进行微调以更新其知识成本高昂。为此，我们提出直接在LLMs的输入中增强知识。具体而言，我们首先根据问题与其关联事实之间的语义相似性，从知识图谱中检索与输入问题相关的事实。随后，将检索到的事实以提示形式前置到输入问题中，再将其输入LLMs以生成答案。我们的框架——知识增强型语言模型提示方法（KAPING）无需模型训练，因此完全零样本。我们在知识图谱问答任务上验证了KAPING框架的性能，该任务旨在基于知识图谱中的事实回答用户问题。实验表明，在多种不同规模的LLMs上，我们的方法在平均性能上比相关零样本基线方法高出最多48%。