Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning. Both encoder-decoder models and prompt-based techniques have shown immense potential for natural language processing and code-based tasks. Over the past several years, many research labs and institutions have invested heavily in high-performance computing, approaching or breaching exascale performance levels. In this paper, we posit that adapting and utilizing such language model-based techniques for tasks in high-performance computing (HPC) would be very beneficial. This study presents our reasoning behind the aforementioned position and highlights how existing ideas can be improved and adapted for HPC tasks.

翻译：近年来，语言模型（LMs），特别是大型语言模型（LLMs），已彻底革新了深度学习领域。编码器-解码器模型和基于提示的技术在自然语言处理和代码任务中展现出巨大潜力。过去数年间，众多研究实验室和机构在高性能计算（HPC）领域投入巨资，已接近或突破百亿亿次性能水平。本文认为，将此类基于语言模型的技术引入并适用于高性能计算（HPC）任务将大有裨益。本研究阐述了上述观点的依据，并重点探讨如何改进现有思路使其适配HPC任务。