The alignment between RNA sequences and structures in foundation models (FMs) has yet to be thoroughly investigated. Existing FMs have struggled to establish sequence-structure alignment, hindering the free flow of genomic information between RNA sequences and structures. In this study, we introduce OmniGenome, an RNA FM trained to align RNA sequences with respect to secondary structures based on structure-contextualised modelling. The alignment enables free and bidirectional mappings between sequences and structures by utilising the flexible RNA modelling paradigm that supports versatile input and output modalities, i.e., sequence and/or structure as input/output. We implement RNA design and zero-shot secondary structure prediction as case studies to evaluate the Seq2Str and Str2Seq mapping capacity of OmniGenome. Results on the EternaV2 benchmark show that OmniGenome solved 74% of puzzles, whereas existing FMs only solved up to 3% of the puzzles due to the oversight of sequence-structure alignment. We leverage four comprehensive in-silico genome modelling benchmarks to evaluate performance across a diverse set of genome downstream tasks, where the results show that OmniGenome achieves state-of-the-art performance on RNA and DNA benchmarks, even without any training on DNA genomes.

翻译：在基础模型中，RNA序列与结构之间的对齐尚未得到深入研究。现有基础模型难以建立序列-结构对齐，阻碍了基因组信息在RNA序列与结构之间的自由流动。本研究提出OmniGenome，这是一种基于结构上下文建模训练而成的RNA基础模型，旨在实现RNA序列与二级结构的对齐。通过采用支持多样化输入输出模态（即序列和/或结构作为输入/输出）的灵活RNA建模范式，该对齐实现了序列与结构之间自由且双向的映射。我们以RNA设计与零样本二级结构预测作为案例研究，评估OmniGenome的Seq2Str与Str2Seq映射能力。在EternaV2基准测试上的结果表明，OmniGenome解决了74%的难题，而现有基础模型由于忽视序列-结构对齐，仅解决了最多3%的难题。我们利用四个综合的计算机基因组建模基准来评估模型在多样化基因组下游任务上的性能，结果显示OmniGenome在RNA和DNA基准测试中均取得了最先进的性能，且未经过任何DNA基因组训练。