The 5' UTR, a regulatory region at the beginning of an mRNA molecule, plays a crucial role in regulating the translation process and impacts the protein expression level. Language models have showcased their effectiveness in decoding the functions of protein and genome sequences. Here, we introduced a language model for 5' UTR, which we refer to as the UTR-LM. The UTR-LM is pre-trained on endogenous 5' UTRs from multiple species and is further augmented with supervised information including secondary structure and minimum free energy. We fine-tuned the UTR-LM in a variety of downstream tasks. The model outperformed the best-known benchmark by up to 42% for predicting the Mean Ribosome Loading, and by up to 60% for predicting the Translation Efficiency and the mRNA Expression Level. The model also applies to identifying unannotated Internal Ribosome Entry Sites within the untranslated region and improves the AUPR from 0.37 to 0.52 compared to the best baseline. Further, we designed a library of 211 novel 5' UTRs with high predicted values of translation efficiency and evaluated them via a wet-lab assay. Experiment results confirmed that our top designs achieved a 32.5% increase in protein production level relative to well-established 5' UTR optimized for therapeutics.

翻译：5' UTR是mRNA分子起始端的调控区域，在调控翻译过程中具有关键作用并影响蛋白质表达水平。语言模型已展现出在解码蛋白质和基因组序列功能方面的有效性。本文提出了一种针对5' UTR的语言模型，我们称之为UTR-LM。该模型基于多个物种的内源性5' UTR进行预训练，并通过包含二级结构和最小自由能在内的监督信息进行增强。我们在多种下游任务中对UTR-LM进行了微调。在预测平均核糖体负载方面，该模型比已知最优基准模型性能提升高达42%；在预测翻译效率和mRNA表达水平方面，性能提升高达60%。该模型还可用于识别非翻译区内未标注的内部核糖体进入位点，与最佳基线相比，将AUPR从0.37提升至0.52。此外，我们设计了一个包含211个具有高翻译效率预测值的全新5' UTR文库，并通过湿实验进行了验证。实验结果证实，与经过治疗优化且广泛应用的5' UTR相比，我们最优设计使蛋白质产量提升了32.5%。