Large language models (LLMs) have shown remarkable performance across a wide range of applications, often outperforming human experts. However, deploying these parameter-heavy models efficiently for diverse inference use cases requires carefully designed hardware platforms with ample computing, memory, and network resources. With LLM deployment scenarios and models evolving at breakneck speed, the hardware requirements to meet SLOs remains an open research question. In this work, we present an analytical tool, GenZ, to study the relationship between LLM inference performance and various platform design parameters. Our analysis provides insights into configuring platforms for different LLM workloads and use cases. We quantify the platform requirements to support SOTA LLMs models like LLaMA and GPT-4 under diverse serving settings. Furthermore, we project the hardware capabilities needed to enable future LLMs potentially exceeding hundreds of trillions of parameters. The trends and insights derived from GenZ can guide AI engineers deploying LLMs as well as computer architects designing next-generation hardware accelerators and platforms. Ultimately, this work sheds light on the platform design considerations for unlocking the full potential of large language models across a spectrum of applications. The source code is available at https://github.com/abhibambhaniya/GenZ-LLM-Analyzer .

翻译：大型语言模型（LLM）在广泛的应用中展现出卓越性能，往往超越人类专家水平。然而，为多样化的推理应用场景高效部署这些参数量庞大的模型，需要精心设计具备充足计算、内存及网络资源的硬件平台。随着LLM部署场景与模型的高速演进，满足服务等级协议（SLO）的硬件需求仍是一个开放的研究问题。本研究提出一种分析工具GenZ，用于探究LLM推理性能与各类平台设计参数之间的关联。我们的分析为不同LLM工作负载与应用场景的平台配置提供了关键见解。我们量化了在多样化服务场景下支持LLaMA、GPT-4等前沿LLM模型所需的平台资源。此外，我们预测了未来参数量可能突破数百万亿的LLM所需硬件能力。基于GenZ推导的趋势与洞见，可为部署LLM的AI工程师以及设计下一代硬件加速器与平台的计算机体系结构研究者提供指导。最终，本研究通过阐明平台设计考量，为释放大型语言模型在全谱系应用中的潜力提供了理论支撑。源代码发布于https://github.com/abhibambhaniya/GenZ-LLM-Analyzer。