Aligning neural activity across subjects offers the promise of discovering shared computational principles and generalizable decoders. However, traditional alignment methods require shared stimuli across subjects, a constraint that limits applicability to naturalistic paradigms with limited or non-overlapping data. We introduce a Multi-Encoder-Decoder Variational Autoencoder (MED-VAE) that achieves cross-subject alignment without shared stimuli by anchoring representations to a common scaffold provided by a pretrained ANN. Using the Natural Scenes Dataset, we show that MED-VAE creates common latent spaces with superior semantic organisation, achieving higher cross-subject alignment than common methods while maintaining robust generalisation to held-out stimuli where traditional methods degrade. Reconstructing from these common spaces back to each subject's original neural space, MED-VAE preserves equal stimulus-driven signal in its cross-subject latent space. Finally, we show that this superior alignment directly enables cross-subject neural prediction, as demonstrated via cross-subject image decoding. In summary, we introduce a framework to identify generalisable common subspaces for cross-subject predictions and downstream tasks, demonstrated here for visual cortex responses to static images.

翻译：跨被试的神经活动对齐有望揭示共享的计算原理和泛化的解码器。然而，传统对齐方法要求被试共享相同刺激，这一限制使其难以应用于数据有限或无重叠刺激的自然场景范式。我们提出一种多编码器-解码器变分自编码器（MED-VAE），通过将神经表征锚定到预训练ANN提供的公共支架上，实现无需共享刺激的跨被试对齐。在自然场景数据集上的实验表明，MED-VAE构建的公共潜空间具有优越的语义组织特性，其跨被试对齐效果优于传统方法，同时能在传统方法失效的保留刺激上保持稳健泛化。通过将公共潜空间重建回各被试的原始神经空间，MED-VAE的跨被试潜空间保留了等量的刺激驱动信号。最终，我们证明这种优越的对齐可直接实现跨被试神经预测——正如跨被试图像解码实验所示。综上，我们提出一个用于识别可泛化公共子空间的框架，适用于跨被试预测及下游任务，并以视觉皮层对静态图像响应为例进行验证。