As Large Language Models (LLMs) become increasingly prevalent in various domains, their ability to process inputs of any length and maintain a degree of memory becomes essential. However, the one-off input of overly long texts is limited, as studies have shown that when input lengths exceed the LLMs' pre-trained text length, there is a dramatic decline in text generation capabilities. Moreover, simply extending the length of pre-training texts is impractical due to the difficulty in obtaining long text data and the substantial memory consumption costs this would entail for LLMs. Recent efforts have employed streaming inputs to alleviate the pressure of excessively long text inputs, but this approach can significantly impair the model's long-term memory capabilities. Motivated by this challenge, we introduce Streaming Infinite Retentive LLM (SirLLM), which allows LLMs to maintain longer memory during infinite-length dialogues without the need for fine-tuning. SirLLM utilizes the Token Entropy metric and a memory decay mechanism to filter key phrases, endowing LLMs with both long-lasting and flexible memory. We designed three distinct tasks and constructed three datasets to measure the effectiveness of SirLLM from various angles: (1) DailyDialog; (2) Grocery Shopping; (3) Rock-Paper-Scissors. Our experimental results robustly demonstrate that SirLLM can achieve stable and significant improvements across different LLMs and tasks, compellingly proving its effectiveness. When having a coversation, "A sir could forget himself," but SirLLM never does! Our code is publicly available at https://github.com/Zoeyyao27/SirLLM

翻译：随着大语言模型（LLMs）在各领域的日益普及，其处理任意长度输入并维持一定记忆能力变得至关重要。然而，一次性输入超长文本存在局限，研究表明当输入长度超过LLMs预训练文本长度时，文本生成能力会急剧下降。此外，由于难以获取长文本数据且会大幅增加LLMs的内存消耗成本，简单延长预训练文本长度并不现实。近期研究采用流式输入缓解超长文本输入带来的压力，但这种方法会显著削弱模型的长期记忆能力。受此挑战驱动，我们提出流式无限记忆保持大语言模型（SirLLM），使LLMs在无需微调的情况下，在无限长度对话中维持更长效记忆。SirLLM利用Token熵指标和记忆衰减机制过滤关键短语，赋予LLMs兼具持久性与灵活性的记忆能力。我们设计了三种差异化任务并构建三个数据集，从多维度衡量SirLLM有效性：（1）日常对话（DailyDialog）；（2）杂货购物（Grocery Shopping）；（3）石头剪刀布（Rock-Paper-Scissors）。实验结果有力证明，SirLLM在不同LLM与任务中均能实现稳定且显著的提升，充分验证其有效性。对话中"君子可能忘怀"，但SirLLM永不遗忘！我们的代码已在https://github.com/Zoeyyao27/SirLLM 开源。