Advances in hybrid bonding and packaging have driven growing interest in 3D DRAM-stacked accelerators with higher memory bandwidth and capacity. As LLMs scale to hundreds of billions or trillions of parameters, distributed inference across multiple 3D chips becomes essential. With cross-stack co-design increasingly critical, we propose DeepStack, an accurate and efficient performance model and tool to enable early-stage system-hardware co-design space exploration (DSE) for distributed 3D-stacked AI systems. At the hardware level, DeepStack captures fine-grained 3D memory semantics such as transaction-aware bandwidth, bank activation constraints, buffering limitations, and thermal-power modeling. At the system level, DeepStack incorporates comprehensive parallelization strategies and execution scheduling for distributed LLM inference. With novel modeling techniques such as dual-stage network abstraction and tile-level compute-communication overlap, we achieve up to 100,000x faster runtime over state-of-the-art simulators at comparable accuracy, cross-validated against our in-house 3D designs, NS-3 backend (2.12%), and vLLM serving on 8xB200 GPUs (12.18%). With hierarchical design space search, DeepStack enables efficient exploration over 2.5x10^14 design points spanning 3D-stacked DRAM layers, DRAM vertical connectivity, interconnect, compute-memory allocation, and distributed scheduling. Compared with baseline designs, DeepStack achieves up to 9.5x higher throughput through co-optimized parallelism and 3D architecture search. Our DSE further reveals that batch size drives a more fundamental architectural divide than the prefill/decode distinction, and that parallelism strategy and hardware architecture are tightly coupled -- incomplete schedule search leads to permanently suboptimal silicon irrecoverable by software tuning. We intend to open source DeepStack to support future research.

翻译：混合键合与封装技术的进步推动了对具有更高内存带宽和容量的3D DRAM堆叠加速器的研究兴趣。随着大语言模型扩展至数千亿或数万亿参数，跨多个3D芯片的分布式推理变得至关重要。基于跨栈协同设计日益重要的趋势，我们提出DeepStack——一种精确高效的性能模型和工具，用于支持分布式3D堆叠AI系统的早期系统-硬件协同设计空间探索。在硬件层面，DeepStack捕获细粒度3D内存语义，如事务感知带宽、存储体激活约束、缓冲限制及热功耗建模。在系统层面，DeepStack集成了分布式LLM推理的全面并行化策略和执行调度。通过双阶段网络抽象和瓦片级计算-通信重叠等新型建模技术，我们实现了相较于最先进模拟器高达100,000倍的运行速度提升，同时保持可比精度——该结果已通过我们自研的3D设计、NS-3后端（2.12%）以及基于8×B200 GPU（12.18%）的vLLM服务端交叉验证。借助分层设计空间搜索，DeepStack能够在覆盖3D堆叠DRAM层、DRAM垂直互连、互连网络、计算-内存分配及分布式调度的2.5×10^14个设计点上实现高效探索。与基准设计相比，DeepStack通过联合优化并行性和3D架构搜索实现了高达9.5倍的吞吐量提升。我们的设计空间探索进一步揭示：批量大小引发的架构分异比预填充/解码阶段的划分更为根本，且并行策略与硬件架构紧密耦合——不完整的调度搜索将导致永久性次优硅设计，且无法通过软件调优弥补。我们计划开源DeepStack以支持未来研究。