The exponential growth of large language models has outpaced the capabilities of traditional CPU and GPU architectures due to the slowdown of Moore's Law. Dataflow AI accelerators present a promising alternative; however, there remains a lack of in-depth performance analysis and standardized benchmarking methodologies for LLM training. We introduce DABench-LLM, the first benchmarking framework designed for evaluating LLM workloads on dataflow-based accelerators. By combining intra-chip performance profiling and inter-chip scalability analysis, DABench-LLM enables comprehensive evaluation across key metrics such as resource allocation, load balance, and resource efficiency. The framework helps researchers rapidly gain insights into underlying hardware and system behaviors, and provides guidance for performance optimizations. We validate DABench-LLM on three commodity dataflow accelerators, Cerebras WSE-2, SambaNova RDU, and Graphcore IPU. Our framework reveals performance bottlenecks and provides specific optimization strategies, demonstrating its generality and effectiveness across a diverse range of dataflow-based AI hardware platforms.

翻译：大语言模型的指数级增长已超越传统CPU和GPU架构的能力边界，其根源在于摩尔定律的放缓。数据流AI加速器提供了一种极具前景的替代方案；然而，目前针对大语言模型训练仍缺少深入的性能分析以及标准化的基准评测方法。我们提出了DABench-LLM，这是首个专为评估数据流加速器上大语言模型工作负载而设计的基准评测框架。通过结合芯片内性能剖析与芯片间可扩展性分析，DABench-LLM能够围绕资源分配、负载均衡及资源效率等关键指标实现全面评估。该框架帮助研究人员快速洞察底层硬件与系统行为，并为性能优化提供指导。我们在三款商用数据流加速器——Cerebras WSE-2、SambaNova RDU和Graphcore IPU上验证了DABench-LLM。我们的框架揭示了性能瓶颈并提供具体优化策略，证明了其在多种基于数据流的AI硬件平台上的通用性与有效性。