Decoding brain imaging data are gaining popularity, with applications in brain-computer interfaces and the study of neural representations. Decoding is typicallysubject-specific and does not generalise well over subjects, due to high amounts ofbetween subject variability. Techniques that overcome this will not only providericher neuroscientific insights but also make it possible for group-level models to out-perform subject-specific models. Here, we propose a method that uses subjectembedding, analogous to word embedding in natural language processing, to learnand exploit the structure in between-subject variability as part of a decoding model,our adaptation of the WaveNet architecture for classification. We apply this to mag-netoencephalography data, where 15 subjects viewed 118 different images, with30 examples per image; to classify images using the entire 1 s window followingimage presentation. We show that the combination of deep learning and subjectembedding is crucial to closing the performance gap between subject- and group-level decoding models. Importantly, group models outperform subject models onlow-accuracy subjects (although slightly impair high-accuracy subjects) and can behelpful for initialising subject models. While we have not generally found group-levelmodels to perform better than subject-level models, the performance of groupmodelling is expected to be even higher with bigger datasets. In order to providephysiological interpretation at the group level, we make use of permutation featureimportance. This provides insights into the spatiotemporal and spectral informationencoded in the models. All code is available on GitHub (https://github.com/ricsinaruto/MEG-group-decode).

翻译：脑成像数据解码正日益流行，其在脑机接口和神经表征研究中有广泛应用。由于受试者间存在高度变异性，解码通常针对特定受试者，难以在受试者间泛化。克服这一问题的技术不仅能提供更丰富的神经科学见解，还可使群体级模型性能超越受试者特异性模型。本文提出一种利用受试者嵌入的方法——类似于自然语言处理中的词嵌入——来学习并利用受试者间变异性的结构特征作为解码模型的一部分，这是我们对WaveNet架构进行分类的改进。我们将该方法应用于脑磁图数据：15名受试者观看118张不同图像，每张图像有30个样本；利用图像呈现后1秒窗口的全部数据对图像进行分类。研究表明，深度学习与受试者嵌入的结合对缩小受试者级与群体级解码模型的性能差距至关重要。重要的是，群体模型在低准确率受试者上的表现优于受试者模型（尽管对高准确率受试者略有削弱），并有助于初始化受试者模型。尽管我们尚未发现群体级模型普遍优于受试者级模型，但预期在更大数据集上群体建模的性能将进一步提升。为提供群体层面的生理学解释，我们采用了置换特征重要性方法，从而揭示模型中编码的时空与频谱信息。所有代码已在GitHub上开源（https://github.com/ricsinaruto/MEG-group-decode）。