Brain decoding aims to interpret and translate neural activity into behaviors. As such, it is imperative that decoding models are able to generalize across variations, such as recordings from different brain sites, distinct sessions, different types of behavior, and a variety of subjects. Current models can only partially address these challenges and warrant the development of pretrained neural transformer models capable to adapt and generalize. In this work, we propose RPNT - Robust Pretrained Neural Transformer, designed to achieve robust generalization through pretraining, which in turn enables effective finetuning given a downstream task. In particular, RPNT unique components include 1) Multidimensional rotary positional embedding (MRoPE) to aggregate experimental metadata such as site coordinates, session name and behavior types; 2) Context-based attention mechanism via convolution kernels operating on global attention to learn local temporal structures for handling non-stationarity of neural population activity; 3) Robust self-supervised learning (SSL) objective with uniform causal masking strategies and contrastive representations. We pretrained two separate versions of RPNT on distinct datasets a) Multi-session, multi-task, and multi-subject microelectrode benchmark; b) Multi-site recordings using high-density Neuropixel 1.0 probes. The datasets include recordings from the dorsal premotor cortex (PMd) and from the primary motor cortex (M1) regions of nonhuman primates (NHPs) as they performed reaching tasks. After pretraining, we evaluated the generalization of RPNT in cross-session, cross-type, cross-subject, and cross-site downstream behavior decoding tasks. Our results show that RPNT consistently achieves and surpasses the decoding performance of existing decoding models in all tasks.

翻译：脑解码旨在解释并将神经活动转化为行为。因此，解码模型必须能够泛化于各种变异，例如来自不同脑区、不同实验阶段、不同行为类型以及不同受试者的记录。现有模型仅能部分应对这些挑战，这促使了能够适应并泛化的预训练神经Transformer模型的开发。在本研究中，我们提出了RPNT——鲁棒预训练神经Transformer，其设计目标是通过预训练实现鲁棒泛化，从而在下游任务中实现有效的微调。具体而言，RPNT的独特组件包括：1）多维旋转位置编码（MRoPE），用于聚合实验元数据，如记录位点坐标、实验阶段名称和行为类型；2）基于上下文的注意力机制，通过作用于全局注意力的卷积核来学习局部时间结构，以处理神经群体活动的非平稳性；3）鲁棒的自监督学习目标，采用均匀因果掩码策略和对比表征。我们在两个不同的数据集上分别预训练了两个版本的RPNT：a）多阶段、多任务、多受试者的微电极基准数据集；b）使用高密度Neuropixel 1.0探针记录的多位点数据集。这些数据集包含非人灵长类动物在执行伸手任务时，来自背侧前运动皮层和初级运动皮层区域的记录。预训练后，我们在跨阶段、跨类型、跨受试者和跨位点的下游行为解码任务中评估了RPNT的泛化能力。我们的结果表明，RPNT在所有任务中均一致达到并超越了现有解码模型的解码性能。