LLM-based agents for industrial asset operations show limited accuracy when reasoning over flat document stores. AssetOpsBench (KDD 2026) establishes that GPT-4 agents achieve 65% on 139 industrial maintenance scenarios backed by CouchDB, YAML, and CSV. It compares LLM orchestration paradigms (Agent-As-Tool vs Plan-Execute) on a fixed data layer; we ask a complementary, orthogonal question: how much does the data model behind the tools affect agent performance? Building on the same scenarios, we introduce a knowledge graph layer (781 nodes, 955 edges, 16 relationship types) and evaluate three architectures: (1) deterministic graph handlers (no LLM) at 99% (137/139); (2) LLM-generated Cypher over the graph at 82-83% with the same GPT-4 model the baseline uses; and (3) the original tool-augmented LLM baseline at 65% (91/139, matching the published KDD 2026 leaderboard ceiling). Our key finding is inverted LLM usage: rather than asking the LLM to reason over raw data, we ask it to generate structured queries from a typed schema. The graph executes deterministically. We additionally contribute 40 graph-native scenarios (multi-hop dependency, vector similarity, PageRank criticality), and evaluate against the expanded HuggingFace AssetOpsBench release (467 scenarios, 6 domains), where deterministic handlers achieve 100% (467/467) with average score 0.848. These results suggest that for structured operational domains, the data layer -- not the LLM orchestration -- is the primary bottleneck, and that knowledge graphs serve as an integration layer between raw industrial data and LLM-based reasoning.
翻译:基于大模型的工业资产运营智能体在处理扁平文档存储时准确性有限。AssetOpsBench(KDD 2026)表明,GPT-4智能体在139个基于CouchDB、YAML和CSV的工业维护场景中准确率为65%。该基准测试在固定数据层上比较了大模型编排范式(工具即智能体与计划-执行);我们提出一个互补且正交的问题:工具背后的数据模型对智能体性能影响有多大?基于相同场景,我们引入知识图谱层(781个节点、955条边、16种关系类型),并评估三种架构:(1)确定性图处理器(无大模型)达到99%(137/139);(2)基于大模型在图上生成Cypher查询(使用与基线相同的GPT-4模型)达到82-83%;(3)原始工具增强大模型基线为65%(91/139,与已发布的KDD 2026排行榜上限一致)。我们的核心发现是逆向使用大模型:不要求大模型对原始数据进行推理,而是让其从类型化模式生成结构化查询。图引擎确定性执行。我们额外贡献了40个图原生场景(多跳依赖、向量相似性、PageRank关键性),并在扩展后的HuggingFace AssetOpsBench版本(467个场景、6个领域)上进行评估,其中确定性处理器达到100%(467/467),平均得分0.848。这些结果表明,对于结构化运维领域,数据层——而非大模型编排——是主要瓶颈,知识图谱可作为原始工业数据与大模型推理之间的集成层。