Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system that uses the advanced language capabilities of LLMs to (1) decide what content to store together in a memory episode, (2) compress those memory episodes into short episodic memories called gist memories, and (3) take actions to look up passages in the original text if ReadAgent needs to remind itself of relevant details to complete a task. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories. These evaluations are performed on three long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum. ReadAgent outperforms the baselines on all three tasks while extending the effective context window by 3-20x.

翻译：当前大型语言模型不仅受限于最大上下文长度，还无法稳健地处理长文本输入。为解决这些局限，我们提出ReadAgent——一种在实验中可将有效上下文长度提升至20倍的LLM代理系统。受人类交互式阅读长文档方式的启发，我们通过利用LLM的先进语言能力，将ReadAgent实现为一种简单的提示系统，其功能包括：(1) 决定如何将内容分组存储为记忆片段；(2) 将这些记忆片段压缩为称为要点记忆的短时情景记忆；(3) 当需要回忆相关细节完成特定任务时，执行操作查阅原始文本中的对应段落。我们基于检索方法、原始长上下文输入以及要点记忆这三类基线方法对ReadAgent进行了评估。这些评估在三个长文档阅读理解任务（QuALITY、NarrativeQA和QMSum）上完成。ReadAgent在所有三项任务中均优于基线方法，同时将有效上下文窗口扩展了3-20倍。