Large language models (LLMs) enhanced by retrieval-augmented generation (RAG) provide effective solutions in various application scenarios. However, developers face challenges in integrating RAG-enhanced LLMs into software systems, due to lack of interface specification, requirements from software context, and complicated system management. In this paper, we manually studied 100 open-source applications that incorporate RAG-enhanced LLMs, and their issue reports. We have found that more than 98% of applications contain multiple integration defects that harm software functionality, efficiency, and security. We have also generalized 19 defect patterns and proposed guidelines to tackle them. We hope this work could aid LLM-enabled software development and motivate future research.

翻译：检索增强生成（RAG）技术增强的大型语言模型（LLM）在各种应用场景中提供了有效的解决方案。然而，由于缺乏接口规范、软件上下文的特定需求以及复杂的系统管理，开发者在将RAG增强的LLM集成到软件系统时面临诸多挑战。本文通过人工分析100个融合了RAG增强型LLM的开源应用及其问题报告，发现超过98%的应用存在多种损害软件功能性、效率与安全性的集成缺陷。我们进一步归纳出19种缺陷模式，并提出了相应的修复指导原则。本研究期望能为LLM驱动的软件开发提供支持，并激发未来的相关研究。