Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. However, their performance may be suboptimal for long-tail or domain-specific tasks due to limited exposure to domain-specific knowledge and vocabulary. Additionally, the lack of transparency of most state-of-the-art (SOTA) LLMs, which can only be accessed via APIs, impedes further fine-tuning with custom data. Moreover, data privacy is a significant concern. To address these challenges, we propose the novel Parametric Knowledge Guiding (PKG) framework, which equips LLMs with a knowledge-guiding module to access relevant knowledge at runtime without altering the LLMs' parameters. Our PKG is based on open-source "white-box" small language models, allowing offline storage of any knowledge that LLMs require. We demonstrate that our PKG framework can enhance the performance of "black-box" LLMs on a range of long-tail and domain-specific downstream tasks requiring factual, tabular, medical, and multimodal knowledge.

翻译：大语言模型凭借其强大的语言理解和生成能力显著推动了自然语言处理的发展。然而，由于对特定领域知识和词汇的接触有限，这些模型在处理长尾或领域特定任务时可能表现欠佳。此外，当前大多数最先进大语言模型因仅能通过应用程序编程接口访问而缺乏透明度，阻碍了利用自定义数据进行进一步微调。同时，数据隐私也成为重要关切。为应对这些挑战，我们提出了创新的参数化知识引导框架，该框架为大语言模型配备知识引导模块，在不修改模型参数的前提下实现运行时相关知识获取。我们的参数化知识引导基于开源"白盒"小语言模型，支持离线存储大语言模型所需的各类知识。实验证明，该框架能够增强"黑盒"大语言模型在需要事实性、表格化、医学及多模态知识的各类长尾与领域特定下游任务中的性能表现。