Large language models (LLMs) have shown remarkable multimodal information processing and reasoning ability. When equipped with tools through function calling and enhanced with retrieval-augmented techniques, compound LLM-based systems can access closed data sources and answer questions about them. However, they still struggle to process and reason over large-scale graph-structure data. We introduce the GDS (Graph Data Science) agent in this technical report. The GDS agent introduces a comprehensive set of graph algorithms as tools, together with preprocessing (retrieval) and postprocessing of algorithm results, in a model context protocol (MCP) server. The server can be used with any modern LLM out-of-the-box. GDS agent allows users to ask any question that implicitly and intrinsically requires graph algorithmic reasoning about their data, and quickly obtain accurate and grounded answers. We introduce new benchmarks that evaluate intermediate tool calls as well as final responses. The results indicate that GDS agent is able to solve a wide spectrum of graph tasks. We also provide detailed case studies for more open-ended tasks and study scenarios where the agent struggles. Finally, we discuss the remaining challenges and the future roadmap.
翻译:大语言模型(LLMs)已展现出卓越的多模态信息处理与推理能力。通过函数调用机制配备工具并借助检索增强技术,基于LLM的复合系统能够访问封闭数据源并回答相关问题。然而,这些系统在处理和推理大规模图结构数据方面仍面临困难。本技术报告介绍了GDS(图数据科学)智能体。该智能体在模型上下文协议(MCP)服务器中集成了一套全面的图算法工具,并包含算法结果的预处理(检索)与后处理流程。该服务器可与任何现代LLM即插即用。GDS智能体允许用户提出任何隐含且本质上需要对其数据进行图算法推理的问题,并快速获得准确且基于事实的答案。我们提出了新的基准测试,用于评估中间工具调用及最终响应。结果表明,GDS智能体能够解决广泛的图任务。我们还针对更开放式的任务提供了详细案例研究,并分析了智能体处理困难的场景。最后,我们探讨了当前面临的挑战及未来发展方向。