Large Language Models (LLMs) face several challenges, including the tendency to produce incorrect outputs, known as hallucination. An effective solution is verifiable text generation, which prompts LLMs to generate content with citations for accuracy verification. However, verifiable text generation is non-trivial due to the focus-shifting phenomenon, the dilemma between the precision and scope in document retrieval, and the intricate reasoning required to discern the relationship between the claim and citations. In this paper, we present VTG, an innovative approach for Verifiable Text Generation with evolving memory and self-reflection. VTG maintains evolving long short-term memory to retain both valuable documents and up-to-date documents. Active retrieval and diverse query generation are utilized to enhance both the precision and scope of the retrieved documents. Furthermore, VTG features a two-tier verifier and an evidence finder, enabling rethinking and reflection on the relationship between the claim and citations. We conduct extensive experiments on five datasets across three knowledge-intensive tasks and the results reveal that VTG significantly outperforms existing baselines.

翻译：大型语言模型面临若干挑战，包括产生错误输出的倾向（即幻觉）。一种有效解决方案是可验证文本生成，即引导模型生成附带引用来源的内容以进行准确性验证。然而，可验证文本生成因注意力偏移现象、文档检索精度与广度间的两难困境，以及判别声明与引用间复杂逻辑关系所需的推理能力而颇具挑战性。本文提出VTG方法——一种基于进化记忆与自反思的创新性可验证文本生成方案。VTG维护进化的长短时记忆机制，同时保留高价值文档与最新文档。通过主动检索与多样化查询生成策略，增强检索文档的精度与广度。此外，VTG配备双层验证器与证据查找器，能够对声明与引用间关系进行重新思考与反思。我们在五个数据集上针对三类知识密集型任务开展全面实验，结果表明VTG显著优于现有基准方法。