Advancements in DNA sequencing technologies have significantly improved our ability to decode genomic sequences. However, the prediction and interpretation of these sequences remain challenging due to the intricate nature of genetic material. Large language models (LLMs) have introduced new opportunities for biological sequence analysis. Recent developments in genomic language models have underscored the potential of LLMs in deciphering DNA sequences. Nonetheless, existing models often face limitations in robustness and application scope, primarily due to constraints in model structure and training data scale. To address these limitations, we present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters. Trained on an expansive dataset comprising 386B bp of eukaryotic DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks. The model adheres to the central dogma of molecular biology, accurately generating protein-coding sequences that translate into proteins structurally analogous to known families. It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles. These capabilities position the GENERator as a pivotal tool for genomic research and biotechnological advancement, enhancing our ability to interpret and predict complex biological systems and enabling precise genomic interventions. Implementation details and supplementary resources are available at https://github.com/GenerTeam/GENERator.

翻译：DNA测序技术的进步显著提升了我们解码基因组序列的能力。然而，由于遗传物质的复杂性，对这些序列的预测和解释仍具挑战性。大语言模型（LLMs）为生物序列分析带来了新的机遇。基因组语言模型的最新进展凸显了LLMs在解读DNA序列方面的潜力。然而，现有模型通常在鲁棒性和应用范围上存在局限，主要源于模型结构和训练数据规模的限制。为应对这些挑战，我们提出了GENERator，一种生成式基因组基础模型，其上下文长度达98k碱基对（bp），参数量为12亿。该模型在包含3860亿bp真核生物DNA的广泛数据集上训练而成，在既有及新提出的基准测试中均展现出最先进的性能。该模型遵循分子生物学的中心法则，能够准确生成可翻译为结构类似于已知家族蛋白质的编码序列。在序列优化方面亦展现出显著潜力，特别是通过提示响应生成具有特定活性谱的增强子序列。这些能力使GENERator成为基因组研究和生物技术发展的关键工具，增强了我们解释和预测复杂生物系统的能力，并实现了精准的基因组干预。实现细节及补充资源详见https://github.com/GenerTeam/GENERator。