Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system that uses the advanced language capabilities of LLMs to (1) decide what content to store together in a memory episode, (2) compress those memory episodes into short episodic memories called gist memories, and (3) take actions to look up passages in the original text if ReadAgent needs to remind itself of relevant details to complete a task. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories. These evaluations are performed on three long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum. ReadAgent outperforms the baselines on all three tasks while extending the effective context window by 3-20x.

翻译：当前大型语言模型（LLMs）不仅受限于最大上下文长度，而且无法鲁棒地处理长输入。为解决这些限制，我们提出ReadAgent，一种LLM智能体系统，在实验中可将有效上下文长度提升至20倍。受人类交互式阅读长文档的启发，我们将ReadAgent实现为一个简单提示系统，利用LLMs的高级语言能力：（1）决定将哪些内容共同存储在记忆片段中；（2）将这些记忆片段压缩为称为要点记忆的短时片段记忆；（3）在需要回忆相关细节以完成任务时，执行操作以查阅原文段落。我们通过检索方法、原始长上下文及要点记忆，将ReadAgent与基线方法进行对比评估。这些评估基于三项长文档阅读理解任务：QuALITY、NarrativeQA和QMSum。ReadAgent在所有三项任务上均优于基线方法，同时将有效上下文窗口扩展3-20倍。