To address increasing compute demand from recent multi-model workloads with heavy models like large language models, we propose to deploy heterogeneous chiplet-based multi-chip module (MCM)-based accelerators. We develop an advanced scheduling framework for heterogeneous MCM accelerators that comprehensively consider complex heterogeneity and inter-chiplet pipelining. Our experiments using our framework on GPT-2 and ResNet-50 models on a 4-chiplet system have shown upto 2.2x and 1.9x increase in throughput and energy efficiency, compared to a monolithic accelerator with an optimized output-stationary dataflow.

翻译：为应对近期以大型语言模型等重型模型为代表的多模型工作负载日益增长的算力需求，我们提出部署基于异构芯粒的多芯片模块（MCM）加速器。我们开发了一种针对异构MCM加速器的高级调度框架，该框架综合考量了复杂异构性与芯粒间流水线技术。在基于4芯粒系统的实验中，针对GPT-2与ResNet-50模型，相较于采用优化输出驻留数据流的单芯片加速器，本框架的吞吐量与能效分别提升了最高2.2倍与1.9倍。