Large Language Models (LLMs) often generate incorrect or outdated information, especially in low-resource settings or when dealing with private data. To address this, Retrieval-Augmented Generation (RAG) uses external knowledge bases (KBs), but these can also suffer from inaccuracies. We introduce STACKFEED, a novel Structured Textual Actor-Critic Knowledge base editing with FEEDback approach that iteratively refines the KB based on expert feedback using a multi-actor, centralized critic reinforcement learning framework. STACKFEED defines a ReACT actor agent on each document to perform structured edits based on document specific targeted instructions. Experimental results showcase that STACKFEED significantly improves KB quality and performance of the RAG system. We evaluate STACKFEED on low-resource programming problems, modified python packaged and factual question-answering tasks.

翻译：大型语言模型（LLMs）常生成错误或过时信息，尤其在低资源场景或处理私有数据时更为明显。为解决此问题，检索增强生成（RAG）采用外部知识库（KBs），但这些知识库同样可能存在不准确问题。本文提出STACKFEED，一种基于反馈的新型结构化文本演员-评论家知识库编辑方法，通过多演员集中式评论家强化学习框架，依据专家反馈迭代优化知识库。STACKFEED在每个文档上定义ReACT演员代理，根据文档特定的目标指令执行结构化编辑。实验结果表明，STACKFEED显著提升了知识库质量及RAG系统性能。我们在低资源编程问题、修改版Python软件包和事实问答任务上对STACKFEED进行了评估。