Retrieval-Augmented Generation (RAG) pipelines are compute-intensive, combining embedding, retrieval, reranking, and large language model (LLM) generation. Running them entirely on-device benefits privacy, latency, and offline use, but the energy cost of CPU inference is a major barrier. We present what is, to our knowledge, the first end-to-end RAG pipeline that runs all neural stages -- embedding, reranking, and LLM generation -- on the Qualcomm Hexagon NPU of the Snapdragon X Elite. Profiling on a Dell XPS 13 laptop, we compare NPU-accelerated RAG against CPU and OpenCL/Adreno GPU baselines on indexing and query workloads. On indexing, the NPU achieves 9.1x higher embedding throughput and 12.3x less system energy. On a 120-query Wikipedia-passage benchmark, it delivers 18.1x faster LLM prefilling, 4.0x lower end-to-end query latency, and 4.0x less system energy than the CPU baseline; the same workload on the integrated GPU is 1.7x slower than CPU and uses 6.5x more energy than the NPU. A GPT-4.1 LLM-as-judge evaluation finds NPU answer quality on par with CPU and GPU within evaluator noise (mean 9.32 vs. 8.95 vs. 9.03 on a 1-10 rubric), with 86.7% of queries scoring identically across all three backends. On the Snapdragon X Elite / Hexagon class of laptop SoC, the NPU thus enables practical, energy-efficient on-device RAG without quality regression -- a sustainable path toward green edge intelligence that we expect to generalize to comparable mobile NPUs (Apple Neural Engine, Intel NPU, MediaTek APU) as their software stacks mature.

翻译：检索增强生成（RAG）流水线计算密集，融合了嵌入、检索、重排序与大语言模型（LLM）生成等环节。将其完全部署于设备端可保障隐私、降低延迟并支持离线使用，但CPU推理的高能耗成为主要障碍。我们提出了业界首个（据我们所知）端到端RAG流水线，其所有神经计算阶段——嵌入、重排序与LLM生成——均在骁龙X Elite平台的高通Hexagon NPU上实现。基于戴尔XPS 13笔记本的效能分析显示，在索引与查询任务中，NPU加速的RAG相较于CPU及OpenCL/Adreno GPU基线方案展现出显著优势：索引阶段NPU的嵌入吞吐量提升9.1倍，系统能耗降低12.3倍；在含120次查询的维基百科段落基准测试中，相较于CPU基线，NPU实现了18.1倍更快的LLM预填充、4.0倍更低的端到端查询延迟及4.0倍的系统能耗降低，而集成GPU在该任务中比CPU慢1.7倍，能耗为NPU的6.5倍。基于GPT-4.1的LLM-as-judge评估表明，在评价者噪声范围内（1-10分制均值：9.32 vs. 8.95 vs. 9.03），NPU生成的答案质量与CPU及GPU相当，其中86.7%的查询在三种后端上的得分完全相同。在骁龙X Elite/Hexagon类笔记本电脑SoC上，NPU使设备端RAG实现了能效优化且无质量退化的实用性突破——这一面向绿色边缘智能的可持续路径，随着苹果神经引擎、英特尔NPU、联发科APU等同类移动NPU软件栈的成熟，有望被广泛推广。