Large language models (LLMs) excel in many natural language processing (NLP) tasks. However, since LLMs can only incorporate new knowledge through training or supervised fine-tuning processes, they are unsuitable for applications that demand precise, up-to-date, and private information not available in the training corpora. This precise, up-to-date, and private information is typically stored in relational databases. Thus, a promising solution is to augment LLMs with the inclusion of relational databases as external memory. This can ensure the timeliness, correctness, and consistency of data, and assist LLMs in performing complex arithmetic operations beyond their inherent capabilities. However, bridging the gap between LLMs and relational databases is challenging. It requires the awareness of databases and data values stored in databases to select correct databases and issue correct SQL queries. Besides, it is necessary for the external memory to be independent of the LLM to meet the needs of real-world applications. We introduce a novel LLM-agnostic memory architecture comprising a database selection memory, a data value memory, and relational databases. And we design an elegant pipeline to retrieve information from it. Besides, we carefully design the prompts to instruct the LLM to maximize the framework's potential. To evaluate our method, we compose a new dataset with various types of questions. Experimental results show that our framework enables LLMs to effectively answer database-related questions, which is beyond their direct ability.

翻译：大型语言模型（LLM）在许多自然语言处理（NLP）任务中表现出色。然而，由于LLM仅能通过训练或有监督微调过程融入新知识，它们不适用于需要精确、最新且未包含在训练语料库中的私有信息的应用场景。这类精确、最新且私有的信息通常存储在关系数据库中。因此，一种有前景的解决方案是将关系数据库作为外部存储器来增强LLM。这可以确保数据的时效性、正确性和一致性，并协助LLM执行超出其固有能力的复杂算术运算。然而，弥合LLM与关系数据库之间的鸿沟具有挑战性。它需要具备对数据库及其中存储数据值的认知，以选择正确的数据库并生成正确的SQL查询。此外，外部存储器必须独立于LLM，以满足实际应用的需求。我们提出了一种新颖的与LLM无关的存储架构，包含数据库选择存储器、数据值存储器和关系数据库，并设计了优雅的信息检索流程。同时，我们精心设计了提示词以指导LLM，从而最大化框架的潜力。为评估我们的方法，我们构建了包含多种问题类型的新数据集。实验结果表明，我们的框架使LLM能够有效回答与数据库相关的问题，这超出了其直接能力范围。