Despite the remarkable ability of large language models (LLMs) in language comprehension and generation, they often suffer from producing factually incorrect information, also known as hallucination. A promising solution to this issue is verifiable text generation, which prompts LLMs to generate content with citations for accuracy verification. However, verifiable text generation is non-trivial due to the focus-shifting phenomenon, the intricate reasoning needed to align the claim with correct citations, and the dilemma between the precision and breadth of retrieved documents. In this paper, we present VTG, an innovative framework for Verifiable Text Generation with evolving memory and self-reflection. VTG introduces evolving long short-term memory to retain both valuable documents and recent documents. A two-tier verifier equipped with an evidence finder is proposed to rethink and reflect on the relationship between the claim and citations. Furthermore, active retrieval and diverse query generation are utilized to enhance both the precision and breadth of the retrieved documents. We conduct extensive experiments on five datasets across three knowledge-intensive tasks and the results reveal that VTG significantly outperforms baselines.

翻译：尽管大语言模型在语言理解与生成方面展现出卓越能力，但它们常会产生事实性错误信息，即所谓的"幻觉"现象。解决此问题的有效方案之一是可验证文本生成，该方法促使模型在生成内容时附带引证以供准确性验证。然而，由于焦点转移现象、需要将声明与正确引证进行对齐的复杂推理过程，以及检索文档在精确性与广度之间的权衡困境，可验证文本生成极具挑战性。本文提出VTG——一种融合演化记忆与自我反思的创新性可验证文本生成框架。VTG引入演化长短时记忆机制，同时保留有价值文档和近期文档；提出配备证据发现器的双层验证器，用于重新审视和反思声明与引证之间的关系；此外，通过主动检索与多样化查询生成策略，提升检索文档的精确性与广度。我们在涵盖三项知识密集型任务的五个数据集上开展大量实验，结果表明VTG显著优于基线方法。