Parameter-efficient finetuning (PEFT) is a key technique for adapting large language models (LLMs) to downstream tasks. In this paper, we study leveraging knowledge graph embeddings to improve the effectiveness of PEFT. We propose a knowledgeable adaptation method called KnowLA. It inserts an adaptation layer into an LLM to integrate the embeddings of entities appearing in the input text. The adaptation layer is trained in combination with LoRA on instruction data. Experiments on six benchmarks with two popular LLMs and three knowledge graphs demonstrate the effectiveness and robustness of KnowLA. We show that \modelname can help activate the relevant parameterized knowledge in an LLM to answer a question without changing its parameters or input prompts.

翻译：参数高效微调（PEFT）是将大型语言模型（LLM）适配至下游任务的关键技术。本文研究如何利用知识图谱嵌入来提升PEFT的效果，并提出了一种名为KnowLA的知识增强适配方法。该方法在LLM中插入适配层，用于整合输入文本中出现的实体嵌入。该适配层与LoRA在指令数据上联合训练。基于两种主流LLM及三个知识图谱的六个基准实验证明，KnowLA具有显著的有效性和鲁棒性。研究表明，该模型可在不改变参数或输入提示的前提下，激活LLM中相关的参数化知识来回答问题。