Domain-specific knowledge bases (KBs) encode vertical expertise and proprietary information that organizations depend on, but curating them at scale is a persistent challenge. Although Large Language Models (LLMs) can draft initial entries efficiently, technical accuracy still requires human expert validation, and reviewing entries one by one at scale is impractical. We present Reflective Agent for Identifier Dictionary (RAID), a novel system that transforms individual expert edits into systematic knowledge updates. Unlike traditional "correct-and-save" paradigms, RAID utilizes a reflective agent to infer the underlying semantic intent behind a single expert edit and propagates that correction across the entire KB through a three-step architecture: Intent Inference, Reflection-based Planning, and User Controlled Execution. We evaluated the reflection and propagation performance on a public dataset and conducted a user study with subject matter experts with proprietary data. The evaluation shows RAID's technical feasibility in capturing expert intent and its potential to scale specialized expertise across industrial knowledge bases.
翻译:领域特定的知识库(KBs)编码了组织依赖的垂直专业知识和专有信息,但大规模策管这些知识库一直是一个持久挑战。尽管大型语言模型(LLMs)能够高效起草初始条目,但技术准确性仍需人类专家验证,而逐条审查大规模条目并不现实。我们提出了一种新颖的系统——用于标识符字典的反思性智能体(RAID),该系统将单个专家编辑转化为系统化的知识更新。与传统的“修正并保存”范式不同,RAID利用一个反思性智能体来推断单个专家编辑背后的潜在语义意图,并通过三步架构(意图推断、基于反思的规划、用户控制执行)将这一修正传播至整个知识库。我们在公开数据集上评估了反思与传播性能,并利用专有数据与领域专家进行了用户研究。评估结果表明,RAID在捕捉专家意图方面具有技术可行性,并具备将专业知识扩展至工业知识库的潜力。