Visual representation learning is ubiquitous in various real-world applications, including visual comprehension, video understanding, multi-modal analysis, human-computer interaction, and urban computing. Due to the emergence of huge amounts of multi-modal heterogeneous spatial/temporal/spatial-temporal data in big data era, the lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. The majority of the existing methods tend to fit the original data/variable distributions and ignore the essential causal relations behind the multi-modal knowledge, which lacks unified guidance and analysis about why modern visual representation learning methods easily collapse into data bias and have limited generalization and cognitive abilities. Inspired by the strong inference ability of human-level agents, recent years have therefore witnessed great effort in developing causal reasoning paradigms to realize robust representation and model learning with good cognitive ability. In this paper, we conduct a comprehensive review of existing causal reasoning methods for visual representation learning, covering fundamental theories, models, and datasets. The limitations of current methods and datasets are also discussed. Moreover, we propose some prospective challenges, opportunities, and future research directions for benchmarking causal reasoning algorithms in visual representation learning. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods, publicly available benchmarks, and consensus-building standards for reliable visual representation learning and related real-world applications more efficiently.
翻译:视觉表示学习在诸多现实世界应用中无处不在,包括视觉理解、视频理解、多模态分析、人机交互和城市计算。大数据时代涌现出海量多模态异质空间/时间/时空数据,现有视觉模型面临可解释性、鲁棒性和分布外泛化能力不足的挑战。大多数现有方法倾向于拟合原始数据/变量分布,而忽略了多模态知识背后的本质因果联系,缺乏对现代视觉表示学习方法为何易陷入数据偏差、泛化与认知能力受限的统一指导与分析。受人类层面智能体强大推理能力的启发,近年来研究者们致力于发展因果推理范式,以实现具有良好认知能力的鲁棒表示与模型学习。本文系统综述了现有面向视觉表示学习的因果推理方法,涵盖基础理论、模型与数据集,并讨论了当前方法与数据集的局限性。此外,我们提出了在视觉表示学习中构建因果推理算法基准的前瞻性挑战、机遇与未来研究方向。本文旨在对这一新兴领域进行全面综述,吸引关注、促进讨论,并强调当务之急是开发新型因果推理方法、公开基准数据集和共识标准,以更高效地实现可靠的视觉表示学习及相关现实应用。