Decoding visual information from human brain activity has seen remarkable advancements in recent research. However, due to the significant variability in cortical parcellation and cognition patterns across subjects, current approaches personalized deep models for each subject, constraining the practicality of this technology in real-world contexts. To tackle the challenges, we introduce Wills Aligner, a robust multi-subject brain representation learner. Our Wills Aligner initially aligns different subjects' brains at the anatomical level. Subsequently, it incorporates a mixture of brain experts to learn individual cognition patterns. Additionally, it decouples the multi-subject learning task into a two-stage training, propelling the deep model and its plugin network to learn inter-subject commonality knowledge and various cognition patterns, respectively. Wills Aligner enables us to overcome anatomical differences and to efficiently leverage a single model for multi-subject brain representation learning. We meticulously evaluate the performance of our approach across coarse-grained and fine-grained visual decoding tasks. The experimental results demonstrate that our Wills Aligner achieves state-of-the-art performance.

翻译：近年来，从人类大脑活动中解码视觉信息的研究取得了显著进展。然而，由于不同被试在皮层分区和认知模式上存在显著差异，当前方法需要为每个被试定制个性化深度模型，这限制了该技术在实际场景中的应用性。为应对这些挑战，我们提出了Wills Aligner——一种鲁棒的多被试脑表征学习器。我们的Wills Aligner首先在解剖层面上对齐不同被试的大脑，随后引入混合脑专家模块以学习个体认知模式。此外，它将多被试学习任务解耦为两阶段训练，使深度模型及其插件网络分别学习被试间的共性知识和多种认知模式。Wills Aligner能够克服解剖差异，并高效利用单一模型进行多被试脑表征学习。我们通过粗粒度和细粒度的视觉解码任务对方法性能进行了细致评估。实验结果表明，Wills Aligner达到了当前最优性能。