Neuroimage processing tasks like segmentation, reconstruction, and registration are central to the study of neuroscience. Robust deep learning strategies and architectures used to solve these tasks are often similar. Yet, when presented with a new task or a dataset with different visual characteristics, practitioners most often need to train a new model, or fine-tune an existing one. This is a time-consuming process that poses a substantial barrier for the thousands of neuroscientists and clinical researchers who often lack the resources or machine-learning expertise to train deep learning models. In practice, this leads to a lack of adoption of deep learning, and neuroscience tools being dominated by classical frameworks. We introduce Neuralizer, a single model that generalizes to previously unseen neuroimaging tasks and modalities without the need for re-training or fine-tuning. Tasks do not have to be known a priori, and generalization happens in a single forward pass during inference. The model can solve processing tasks across multiple image modalities, acquisition methods, and datasets, and generalize to tasks and modalities it has not been trained on. Our experiments on coronal slices show that when few annotated subjects are available, our multi-task network outperforms task-specific baselines without training on the task.

翻译：神经影像处理任务，如分割、重建和配准，是神经科学研究的核心。用于解决这些任务的稳健深度学习策略和架构通常具有相似性。然而，面对新任务或视觉特征不同的数据集时，从业者往往需要训练新模型或对现有模型进行微调。这一过程耗时且技术要求高，对数千名缺乏深度学习模型训练资源或机器学习专业知识的神经科学家和临床研究人员构成了实质性障碍。实践中，这导致深度学习采用率不足，神经科学工具仍以经典框架为主导。我们提出神经化器（Neuralizer），这是一个无需重训练或微调即可泛化到先前未见神经影像任务和模态的单一模型。任务无需先验已知，泛化在推理过程中的单次前向传播中完成。该模型可跨多种图像模态、采集方法和数据集处理任务，并能泛化到其未训练过的任务和模态。我们在冠状切片上的实验表明，当标注受试者样本有限时，我们的多任务网络无需针对特定任务训练即可超越任务相关基线模型。