In the era of Large Language Models (LLMs) with their advanced capabilities, a unique opportunity arises to develop LLM-based digital assistant tools that can support software developers by facilitating comprehensive reasoning about software dependencies and open-source libraries before importing them. This reasoning process is daunting, mandating multiple specialized tools and dedicated expertise, each focusing on distinct aspects (e.g., security analysis tools may overlook design flaws such as circular dependencies, which hinder software maintainability). Creating a significant bottleneck in the software development lifecycle. In this paper, we introduce DepsRAG, a multi-agent framework designed to assist developers in reasoning about software dependencies. DepsRAG first constructs a comprehensive Knowledge Graph (KG) that includes both direct and transitive dependencies. Developers can interact with DepsRAG through a conversational interface, posing queries about the dependencies. DepsRAG employs Retrieval-Augmented Generation (RAG) to enhance these queries by retrieving relevant information from the KG as well as external sources, such as the Web and vulnerability databases, thus demonstrating its adaptability to novel scenarios. DepsRAG incorporates a Critic-Agent feedback loop to ensure the accuracy and clarity of LLM-generated responses. We evaluated DepsRAG using GPT-4-Turbo and Llama-3 on three multi-step reasoning tasks, observing a threefold increase in accuracy with the integration of the Critic-Agent mechanism. DepsRAG demo and implementation are available: https://github.com/Mohannadcse/DepsRAG.
翻译:在大语言模型(LLMs)凭借其先进能力崛起的时代,一个独特的机遇随之出现:开发基于LLM的数字助手工具,以支持软件开发者在导入软件依赖项和开源库之前,进行全面的推理。这一推理过程令人望而生畏,它需要多种专用工具和专门知识,每种工具和知识都侧重于不同的方面(例如,安全分析工具可能忽略诸如循环依赖等设计缺陷,而这些缺陷会损害软件的可维护性),从而在软件开发生命周期中造成了显著的瓶颈。本文中,我们介绍了DepsRAG,一个旨在辅助开发者进行软件依赖推理的多智能体框架。DepsRAG首先构建一个包含直接和传递依赖项的全面知识图谱(KG)。开发者可以通过对话界面与DepsRAG交互,提出关于依赖项的查询。DepsRAG采用检索增强生成(RAG)技术,通过从KG以及外部来源(如网络和漏洞数据库)检索相关信息来增强这些查询,从而展示了其对新颖场景的适应能力。DepsRAG集成了一个批评者-智能体反馈循环,以确保LLM生成响应的准确性和清晰度。我们在三个多步推理任务上使用GPT-4-Turbo和Llama-3评估了DepsRAG,观察到在集成批评者-智能体机制后,准确率提高了三倍。DepsRAG演示和实现可访问:https://github.com/Mohannadcse/DepsRAG。