The development of foundation models for functional magnetic resonance imaging (fMRI) time series holds significant promise for predicting phenotypes related to disease and cognition. Current models, however, are often trained using a mask-and-reconstruct objective on small brain regions. This focus on low-level information leads to representations that are sensitive to noise and temporal fluctuations, necessitating extensive fine-tuning for downstream tasks. We introduce Brain-Semantoks, a self-supervised framework designed specifically to learn abstract representations of brain dynamics. Its architecture is built on two core innovations: a semantic tokenizer that aggregates noisy regional signals into robust tokens representing functional networks, and a self-distillation objective that enforces representational stability across time. We show that this objective is stabilized through a novel training curriculum, ensuring the model robustly learns meaningful features from low signal-to-noise time series. We demonstrate that learned representations enable strong performance on a variety of downstream tasks even when only using a linear probe. Furthermore, we provide comprehensive scaling analyses indicating more unlabeled data reliably results in out-of-distribution performance gains without domain adaptation.

翻译：功能磁共振成像（fMRI）时间序列基础模型的开发，对于预测与疾病和认知相关的表型具有重要前景。然而，当前模型通常是在小脑区域上使用掩码-重建目标进行训练的。这种对低层次信息的关注导致所得表征对噪声和时间波动敏感，在下游任务中需要进行大量微调。我们提出了Brain-Semantoks，这是一个专门为学习脑动态的抽象表征而设计的自监督框架。其架构基于两项核心创新：一个语义标记器，它将嘈杂的区域信号聚合成代表功能网络的鲁棒标记；以及一个自蒸馏目标，它强制表征在时间维度上保持稳定性。我们证明，通过一种新颖的训练课程设计，该目标得以稳定，确保模型能够从低信噪比的时间序列中稳健地学习有意义的特征。我们证明了学习到的表征即使仅使用线性探针，也能在各种下游任务上实现强大的性能。此外，我们提供了全面的缩放分析，表明更多的无标签数据能够可靠地带来分布外性能的提升，而无需进行领域适应。