Deep learning is leading to major advances in the realm of brain decoding from functional Magnetic Resonance Imaging (fMRI). However, the large inter-subject variability in brain characteristics has limited most studies to train models on one subject at a time. Consequently, this approach hampers the training of deep learning models, which typically requires very large datasets. Here, we propose to boost brain decoding by aligning brain responses to videos and static images across subjects. Compared to the anatomically-aligned baseline, our method improves out-of-subject decoding performance by up to 75%. Moreover, it also outperforms classical single-subject approaches when fewer than 100 minutes of data is available for the tested subject. Furthermore, we propose a new multi-subject alignment method, which obtains comparable results to that of classical single-subject approaches while improving out-of-subject generalization. Finally, we show that this method aligns neural representations in accordance with brain anatomy. Overall, this study lays the foundations for leveraging extensive neuroimaging datasets and enhancing the decoding of individuals with a limited amount of brain recordings.

翻译：深度学习正推动从功能磁共振成像(fMRI)进行脑解码领域的重大进展。然而，大脑特征的受试者间巨大变异性限制了大多数研究仅能在单个被试上训练模型。这种策略进而阻碍了通常需要庞大数据集的深度学习模型的训练。本文提出通过跨被试对齐脑对视频和静态图像的响应来增强脑解码能力。与基于解剖学对齐的基线方法相比，我们的方法将跨被试解码性能提升高达75%。此外，当目标被试可用数据少于100分钟时，该方法还优于经典的单被试方法。进一步地，我们提出了一种新的多被试对齐方法，在提升跨被试泛化能力的同时，获得了与经典单被试方法相当的结果。最后，我们证实该方法能按照大脑解剖结构对齐神经表征。总体而言，本研究为利用大规模神经影像数据集并增强对有限脑记录个体的解码能力奠定了基础。