In the era of Large Language Models (LLMs) with their advanced capabilities, a unique opportunity arises to develop LLM-based digital assistant tools that can support software developers by facilitating comprehensive reasoning about software dependencies and open-source libraries before importing them. This reasoning process is daunting, mandating multiple specialized tools and dedicated expertise, each focusing on distinct aspects (e.g., security analysis tools may overlook design flaws such as circular dependencies, which hinder software maintainability). Creating a significant bottleneck in the software development lifecycle. In this paper, we introduce DepsRAG, a multi-agent framework designed to assist developers in reasoning about software dependencies. DepsRAG first constructs a comprehensive Knowledge Graph (KG) that includes both direct and transitive dependencies. Developers can interact with DepsRAG through a conversational interface, posing queries about the dependencies. DepsRAG employs Retrieval-Augmented Generation (RAG) to enhance these queries by retrieving relevant information from the KG as well as external sources, such as the Web and vulnerability databases, thus demonstrating its adaptability to novel scenarios. DepsRAG incorporates a Critic-Agent feedback loop to ensure the accuracy and clarity of LLM-generated responses. We evaluated DepsRAG using GPT-4-Turbo and Llama-3 on three multi-step reasoning tasks, observing a threefold increase in accuracy with the integration of the Critic-Agent mechanism. DepsRAG demo and implementation are available: \url{https://github.com/Mohannadcse/DepsRAG}.
翻译:在具备先进能力的大语言模型(LLM)时代,一个独特的机遇随之出现:开发基于LLM的数字助理工具,以支持软件开发者在导入软件依赖和开源库之前,对其进行全面的推理分析。这一推理过程极具挑战性,需要依赖多种专用工具和专业领域知识,而每种工具通常只关注特定方面(例如,安全分析工具可能忽略诸如循环依赖这类影响软件可维护性的设计缺陷),从而在软件开发生命周期中形成了一个显著的瓶颈。本文提出DepsRAG,一个旨在辅助开发者进行软件依赖推理的多智能体框架。DepsRAG首先构建一个包含直接依赖和传递依赖的全面知识图谱(KG)。开发者可以通过对话界面与DepsRAG交互,提出关于依赖项的查询。DepsRAG采用检索增强生成(RAG)技术,通过从KG以及外部来源(如网络和漏洞数据库)检索相关信息来增强查询,从而展示了其对新场景的适应能力。DepsRAG还集成了一个批评者-智能体反馈循环,以确保LLM生成响应的准确性和清晰度。我们使用GPT-4-Turbo和Llama-3在三个多步推理任务上对DepsRAG进行了评估,观察到在集成批评者-智能体机制后,准确率提升了三倍。DepsRAG的演示和实现可访问:\url{https://github.com/Mohannadcse/DepsRAG}。