The 5' UTR, a regulatory region at the beginning of an mRNA molecule, plays a crucial role in regulating the translation process and impacts the protein expression level. Language models have showcased their effectiveness in decoding the functions of protein and genome sequences. Here, we introduced a language model for 5' UTR, which we refer to as the UTR-LM. The UTR-LM is pre-trained on endogenous 5' UTRs from multiple species and is further augmented with supervised information including secondary structure and minimum free energy. We fine-tuned the UTR-LM in a variety of downstream tasks. The model outperformed the best-known benchmark by up to 42% for predicting the Mean Ribosome Loading, and by up to 60% for predicting the Translation Efficiency and the mRNA Expression Level. The model also applies to identifying unannotated Internal Ribosome Entry Sites within the untranslated region and improves the AUPR from 0.37 to 0.52 compared to the best baseline. Further, we designed a library of 211 novel 5' UTRs with high predicted values of translation efficiency and evaluated them via a wet-lab assay. Experiment results confirmed that our top designs achieved a 32.5% increase in protein production level relative to well-established 5' UTR optimized for therapeutics.

翻译：5' UTR作为mRNA分子起始端的调控区域，在调控翻译过程中发挥关键作用，并影响蛋白质表达水平。语言模型在解码蛋白质和基因组序列功能方面已展现出有效性。本文提出了一种面向5' UTR的语言模型，命名为UTR-LM。该模型基于多物种内源性5' UTR进行预训练，并通过二级结构和最小自由能等监督信息进行增强。我们在多种下游任务中对UTR-LM进行了微调。在预测平均核糖体负载方面，该模型较已知最佳基准性能提升高达42%；在预测翻译效率和mRNA表达水平方面，性能提升高达60%。该模型还可用于识别非翻译区中未注释的内部核糖体进入位点，相较最优基线方法将AUPR从0.37提升至0.52。此外，我们设计了包含211条具有高预测翻译效率值的全新5' UTR文库，并通过湿实验验证。实验结果表明，相较于经过治疗用途优化的成熟5' UTR，我们的最优设计使蛋白质产量提升32.5%。