Large language models (LLMs) are emerging as few-shot learners capable of handling a variety of tasks, including comprehension, planning, reasoning, question answering, arithmetic calculations, and more. At the core of these capabilities is LLMs' proficiency in representing and understanding structural or semi-structural data, such as tables and graphs. Numerous studies have demonstrated that reasoning on tabular data or graphs is not only feasible for LLMs but also gives a promising research direction which treats these data as in-context data. The lightweight and human readable characteristics of in-context database can potentially make it an alternative for the traditional database in typical RAG (Retrieval Augmented Generation) settings. However, almost all current work focuses on static in-context data, which does not allow dynamic update. In this paper, to enable dynamic database update, delta encoding of database is proposed. We explore how data stored in traditional RDBMS can be encoded as in-context text and evaluate LLMs' proficiency for CRUD (Create, Read, Update and Delete) operations on in-context databases. A benchmark named InConDB is presented and extensive experiments are conducted to show the performance of different language models in enabling in-context database by varying the database encoding method, prompting method, operation type and input data distribution, revealing both the proficiency and limitations.

翻译：大型语言模型（LLMs）正逐渐成为能够处理多种任务的少样本学习器，包括理解、规划、推理、问答、算术计算等。这些能力的核心在于LLMs在表示和理解结构或半结构数据（如表和图表）方面的熟练程度。大量研究表明，在表格数据或图表上进行推理不仅对LLMs是可行的，而且是一个有前景的研究方向，将这些数据视为上下文内数据。上下文内数据库的轻量级和人类可读特性，可能使其在典型的RAG（检索增强生成）设置中成为传统数据库的替代方案。然而，目前几乎所有工作都集中在静态的上下文内数据上，不允许动态更新。在本文中，为了实现动态数据库更新，提出了数据库的增量编码方法。我们探讨了存储在传统关系数据库管理系统（RDBMS）中的数据如何被编码为上下文内文本，并评估了LLMs在上下文内数据库上执行CRUD（创建、读取、更新和删除）操作的熟练程度。提出了一个名为InConDB的基准测试，并进行了大量实验，通过改变数据库编码方法、提示方法、操作类型和输入数据分布，展示了不同语言模型在实现上下文内数据库方面的性能，揭示了其熟练程度和局限性。