Neuroimage processing tasks like segmentation, reconstruction, and registration are central to the study of neuroscience. Robust deep learning strategies and architectures used to solve these tasks are often similar. Yet, when presented with a new task or a dataset with different visual characteristics, practitioners most often need to train a new model, or fine-tune an existing one. This is a time-consuming process that poses a substantial barrier for the thousands of neuroscientists and clinical researchers who often lack the resources or machine-learning expertise to train deep learning models. In practice, this leads to a lack of adoption of deep learning, and neuroscience tools being dominated by classical frameworks. We introduce Neuralizer, a single model that generalizes to previously unseen neuroimaging tasks and modalities without the need for re-training or fine-tuning. Tasks do not have to be known a priori, and generalization happens in a single forward pass during inference. The model can solve processing tasks across multiple image modalities, acquisition methods, and datasets, and generalize to tasks and modalities it has not been trained on. Our experiments on coronal slices show that when few annotated subjects are available, our multi-task network outperforms task-specific baselines without training on the task.

翻译：神经影像处理任务，如分割、重建和配准，是神经科学研究的核心。解决这些任务所采用的稳健深度学习策略和架构往往具有相似性。然而，当面临一项新任务或视觉特征不同的数据集时，从业者大多需要训练新模型或微调现有模型。这一过程耗时巨大，给成千上万缺乏资源或机器学习专业知识来训练深度学习模型的神经科学家和临床研究人员带来了实质性障碍。在实践中，这导致深度学习应用不足，而神经科学工具仍以经典框架为主导。我们提出了Neuralizer，这是一个单一模型，能够泛化至先前未见过的神经影像任务和模态，而无需重新训练或微调。任务无需预先知晓，泛化在推理过程中的单次前向传播中完成。该模型可跨多种图像模态、采集方法和数据集解决处理任务，并泛化至其未训练过的任务和模态。我们在冠状切片上的实验表明，当标注样本数量有限时，我们的多任务网络能在未针对该任务进行训练的情况下，超越特定任务基线。