This paper reviews concepts, modeling approaches, and recent findings along a spectrum of different levels of abstraction of neural network models including generalization across (1) Samples, (2) Distributions, (3) Domains, (4) Tasks, (5) Modalities, and (6) Scopes. Strategies for (1) sample generalization from training to test data are discussed, with suggestive evidence presented that, at least for the ImageNet dataset, popular classification models show substantial overfitting. An empirical example and perspectives from statistics highlight how models' (2) distribution generalization can benefit from consideration of causal relationships and counterfactual scenarios. Transfer learning approaches and results for (3) domain generalization are summarized, as is the wealth of domain generalization benchmark datasets available. Recent breakthroughs surveyed in (4) task generalization include few-shot meta-learning approaches and the emergence of transformer-based foundation models such as those used for language processing. Studies performing (5) modality generalization are reviewed, including those that integrate image and text data and that apply a biologically-inspired network across olfactory, visual, and auditory modalities. Higher-level (6) scope generalization results are surveyed, including graph-based approaches to represent symbolic knowledge in networks and attribution strategies for improving networks' explainability. Additionally, concepts from neuroscience are discussed on the modular architecture of brains and the steps by which dopamine-driven conditioning leads to abstract thinking.

翻译：本文综述了神经网络模型在不同抽象层次上的概念、建模方法及最新发现，涵盖以下六个方面的泛化能力：(1) 样本泛化，(2) 分布泛化，(3) 领域泛化，(4) 任务泛化，(5) 模态泛化，以及(6) 范围泛化。针对(1)从训练数据到测试数据的样本泛化策略进行了探讨，并提供了提示性证据表明——至少在ImageNet数据集上——主流分类模型表现出显著的过拟合现象。通过实证案例和统计学视角，本文阐述了模型在(2)分布泛化中如何通过考虑因果关系和反事实场景获得提升。系统总结了(3)领域泛化的迁移学习方法与成果，以及当前可用的丰富领域泛化基准数据集。在(4)任务泛化方面，综述了包括小样本元学习方法在内的最新突破，以及基于Transformer的基础模型（如用于语言处理的模型）的兴起。回顾了涉及(5)模态泛化的研究，包括图像与文本数据的融合研究，以及将受生物启发的网络应用于嗅觉、视觉和听觉模态的工作。调研了更高层次的(6)范围泛化成果，涵盖基于图的方法（用于在网络中表示符号知识）和提升网络可解释性的归因策略。此外，还结合神经科学概念探讨了大脑的模块化架构，以及多巴胺驱动条件反射实现抽象思维的演进过程。