Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system that uses the advanced language capabilities of LLMs to (1) decide what content to store together in a memory episode, (2) compress those memory episodes into short episodic memories called gist memories, and (3) take actions to look up passages in the original text if ReadAgent needs to remind itself of relevant details to complete a task. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories. These evaluations are performed on three long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum. ReadAgent outperforms the baselines on all three tasks while extending the effective context window by 3.5-20x.

翻译：当前的大型语言模型（LLMs）不仅受限于最大上下文长度，而且无法稳健地处理长输入。为应对这些局限，我们提出了ReadAgent——一种LLM智能体系统，在我们的实验中其有效上下文长度可提升至20倍。受人类交互式阅读长文档方式的启发，我们将ReadAgent实现为一种简单的提示系统，该系统利用LLMs的高级语言能力实现以下功能：（1）决定将哪些内容共同存储于记忆片段中；（2）将这些记忆片段压缩为称为要点记忆的简短情景记忆；（3）若ReadAgent需要回忆相关细节以完成任务，则执行检索原始文本段落的行为。我们在三个长文档阅读理解任务（QuALITY、NarrativeQA和QMSum）上，通过对比基于检索方法的基线模型、使用原始长上下文模型以及使用要点记忆模型，对ReadAgent进行评估。实验结果表明，ReadAgent在有效上下文窗口扩展3.5-20倍的同时，在所有三项任务上均优于基线模型。