Retrieval-Augmented Generation (RAG) delivers substantial value in knowledge-intensive applications. However, its generated responses often lack transparent reasoning paths that trace back to source evidence from retrieved documents. This opacity not only compromises the interpretability of the output but also limits the model's ability to fully exploit the provided context. To address this, we propose TRACE (Transparent RAG with evidenCE tracing), a framework designed to enhance evidence traceability in Large Language Models (LLMs) through reinforcement learning (RL). TRACE guides LLMs to produce structured outputs with explicit evidence citations by prompting and rewarding evidence relevance and proper formatting, alongside accuracy, to optimize structured traceability. To ensure training stability with multiple reward signals, we further introduce an adaptive strategy for merging rewards and adopt a stabilized KL-divergence estimator. Experiments on three multi-hop QA datasets using Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct show that TRACE achieves both transparent, evidence-attributed outputs and accuracy improvements of 10-30%. The resulting performance is comparable to advanced commercial LLMs (e.g., OpenAI o1, DeepSeek-R1). Further analyses demonstrate strong generalization capabilities to unseen tasks. Our code is publicly available now.

翻译：检索增强生成（RAG）在知识密集型应用中展现出重要价值。然而，其生成的回答往往缺乏能够追溯至检索文档中源证据的透明推理路径。这种不透明性不仅损害了输出的可解释性，也限制了模型充分利用所提供上下文的能力。为解决此问题，我们提出了TRACE（具有证据追溯的透明化RAG），这是一个通过强化学习（RL）增强大语言模型（LLMs）中证据可追溯性的框架。TRACE通过提示和奖励证据相关性及正确格式（同时兼顾准确性）来引导LLMs生成带有明确证据引用的结构化输出，从而优化结构化可追溯性。为确保在多重奖励信号下的训练稳定性，我们进一步引入了自适应奖励融合策略，并采用了稳定的KL散度估计器。在三个多跳问答数据集上使用Qwen2.5-7B-Instruct和Llama-3.1-8B-Instruct进行的实验表明，TRACE既能实现透明、具备证据归属的输出，又能将准确率提升10-30%。其最终性能可与先进的商业LLMs（如OpenAI o1、DeepSeek-R1）相媲美。进一步的分析展示了其对未见任务的强大泛化能力。我们的代码现已公开。