Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning. Both encoder-decoder models and prompt-based techniques have shown immense potential for natural language processing and code-based tasks. Over the past several years, many research labs and institutions have invested heavily in high-performance computing, approaching or breaching exascale performance levels. In this paper, we posit that adapting and utilizing such language model-based techniques for tasks in high-performance computing (HPC) would be very beneficial. This study presents our reasoning behind the aforementioned position and highlights how existing ideas can be improved and adapted for HPC tasks.

翻译：近年来，语言模型尤其是大型语言模型彻底改变了深度学习领域。编码器-解码器模型与提示技术均在自然语言处理和代码型任务中展现出巨大潜力。过去数年间，众多研究实验室与机构在高性能计算领域投入巨资，已接近或突破百亿亿次性能水平。本文认为，将此类基于语言模型的技术适配并应用于高性能计算任务将十分有益。本研究阐述了上述主张的论证依据，并揭示了现有方法可如何改进并适配于HPC任务。