In customer service technical support, swiftly and accurately retrieving relevant past issues is critical for efficiently resolving customer inquiries. The conventional retrieval methods in retrieval-augmented generation (RAG) for large language models (LLMs) treat a large corpus of past issue tracking tickets as plain text, ignoring the crucial intra-issue structure and inter-issue relations, which limits performance. We introduce a novel customer service question-answering method that amalgamates RAG with a knowledge graph (KG). Our method constructs a KG from historical issues for use in retrieval, retaining the intra-issue structure and inter-issue relations. During the question-answering phase, our method parses consumer queries and retrieves related sub-graphs from the KG to generate answers. This integration of a KG not only improves retrieval accuracy by preserving customer service structure information but also enhances answering quality by mitigating the effects of text segmentation. Empirical assessments on our benchmark datasets, utilizing key retrieval (MRR, Recall@K, NDCG@K) and text generation (BLEU, ROUGE, METEOR) metrics, reveal that our method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU. Our method has been deployed within LinkedIn's customer service team for approximately six months and has reduced the median per-issue resolution time by 28.6%.

翻译：在客户服务技术支持中，快速准确地检索相关历史问题对于高效解决客户咨询至关重要。大语言模型（LLM）在检索增强生成（RAG）中的传统检索方法将海量历史工单视为纯文本，忽略了工单内部结构与工单间关系的关键信息，从而限制了性能表现。我们提出一种融合RAG与知识图谱（KG）的新型客服问答方法。该方法利用历史工单构建知识图谱用于检索，保留了工单内部结构与工单间关系。在问答阶段，该方法解析消费者查询，从知识图谱中检索相关子图以生成答案。这种知识图谱的融合不仅通过保留客服结构信息提升了检索精度，而且通过减轻文本分割的影响增强了回答质量。基于自建基准数据集，利用关键检索指标（MRR、Recall@K、NDCG@K）和文本生成指标（BLEU、ROUGE、METEOR）的实验评估表明，本方法在MRR指标上较基线提升77.6%，BLEU指标提升0.32。该方法已在领英客户服务团队内部署约六个月，将单问题中位解决时间降低了28.6%。