Multi-chiplet GPUs split memory into local and remote HBM regions across a silicon interposer, and reducing the remote HBM traffic is crucial for the performance and energy efficiency of multi-chiplet GPUs. For general matrix multiplication (GEMM), the dominant operator in large language models (LLMs), the resulting inter-chiplet traffic depends strongly on kernel choices such as operand layout, CTA traversal order, and data placement, and the optimal strategy to minimize remote accesses is nontrivial. We present a fast, functional, tile-level locality simulator that models CTA scheduling, per-chiplet L2 caches, and local/remote HBM accesses to evaluate a full-size LLM GEMM configuration. Across representative LLM GEMMs, the simulator shows that remote traffic varies by up to 58x across the design space for the same GEMM dimensions. Moreover, using the simulator as feedback, an agentic AI discovers that a 2D block-swizzle CTA traversal reduces remote traffic over the best 1D traversal by up to 5.1x under round-robin placement, identifying CTA traversal order as a first-order, GEMM-dependent design knob for inter-chiplet traffic.

翻译：多芯片GPU通过硅中介层将内存划分为本地和远程HBM区域，减少远程HBM流量对于多芯片GPU的性能和能效至关重要。对于大语言模型（LLM）中的主导算子——通用矩阵乘法（GEMM），由此产生的跨芯片流量强烈依赖于内核选择（如操作数布局、CTA遍历顺序和数据放置方式），而最小化远程访问的最优策略并非显而易见。本文提出一种快速、功能性、基于瓦片级别的局部性模拟器，该模拟器对CTA调度、每芯片L2缓存以及本地/远程HBM访问进行建模，以评估全尺寸LLM GEMM配置。在代表性LLM GEMM中，模拟器显示：对于相同GEMM维度，设计空间内的远程流量差异可达58倍。此外，利用该模拟器作为反馈，一种自主AI发现：在轮询放置策略下，二维块交错CTA遍历相较于最优一维遍历可将远程流量降低高达5.1倍，从而确定CTA遍历顺序是影响跨芯片流量的首要且依赖GEMM的设计参数。