Visual decoding from brain signals is a key challenge at the intersection of computer vision and neuroscience, requiring methods that bridge neural representations and computational models of vision. A field-wide goal is to achieve generalizable, cross-subject models. A major obstacle towards this goal is the substantial variability in neural representations across individuals, which has so far required training bespoke models or fine-tuning separately for each subject. To address this challenge, we introduce a meta-optimized approach for semantic visual decoding from fMRI that generalizes to novel subjects without any fine-tuning. By simply conditioning on a small set of image-brain activation examples from the new individual, our model rapidly infers their unique neural encoding patterns to facilitate robust and efficient visual decoding. Our approach is explicitly optimized for in-context learning of the new subject's encoding model and performs decoding by hierarchical inference, inverting the encoder. First, for multiple brain regions, we estimate the per-voxel visual response encoder parameters by constructing a context over multiple stimuli and responses. Second, we construct a context consisting of encoder parameters and response values over multiple voxels to perform aggregated functional inversion. We demonstrate strong cross-subject and cross-scanner generalization across diverse visual backbones without retraining or fine-tuning. Moreover, our approach requires neither anatomical alignment nor stimulus overlap. This work is a critical step towards a generalizable foundation model for non-invasive brain decoding.

翻译：从大脑信号中进行视觉解码是计算机视觉与神经科学交叉领域的关键挑战，需要能够桥接神经表征与视觉计算模型的方法。该领域的核心目标是实现可泛化的跨被试模型。实现这一目标的主要障碍在于个体间神经表征存在显著差异，目前需要针对每个被试单独训练定制模型或进行微调。为解决该问题，我们提出一种基于元优化的fMRI语义视觉解码方法，无需任何微调即可泛化至新被试。通过仅依赖新个体少量图像-脑激活示例作为条件，我们的模型能快速推断其独特的神经编码模式，从而实现鲁棒且高效的视觉解码。该方法专门针对新被试编码模型的上下文学习进行优化，并通过层级推理（逆向编码器）完成解码。首先，针对多个脑区，我们通过构建跨多个刺激与响应的上下文来估计每个体素的视觉响应编码器参数。其次，我们构建包含编码器参数与跨多体素响应值的上下文，进行聚合功能逆推。实验表明，该方法在多种视觉主干网络上无需重训练或微调即展现出强大的跨被试与跨扫描仪泛化能力。此外，我们的方法既不需要解剖对齐，也不依赖刺激重叠。这项工作向构建可泛化的无创脑解码基础模型迈出了关键一步。