2D image understanding is a complex problem within Computer Vision, but it holds the key to providing human level scene comprehension. It goes further than identifying the objects in an image, and instead it attempts to understand the scene. Solutions to this problem form the underpinning of a range of tasks, including image captioning, Visual Question Answering (VQA), and image retrieval. Graphs provide a natural way to represent the relational arrangement between objects in an image, and thus in recent years Graph Neural Networks (GNNs) have become a standard component of many 2D image understanding pipelines, becoming a core architectural component especially in the VQA group of tasks. In this survey, we review this rapidly evolving field and we provide a taxonomy of graph types used in 2D image understanding approaches, a comprehensive list of the GNN models used in this domain, and a roadmap of future potential developments. To the best of our knowledge, this is the first comprehensive survey that covers image captioning, visual question answering, and image retrieval techniques that focus on using GNNs as the main part of their architecture.
翻译:二维图像理解是计算机视觉中的一个复杂问题,但它为提供人类级别的场景理解能力奠定了基础。该问题不仅局限于识别图像中的物体,更试图理解整个场景。对此问题的解决方案支撑着多项任务,包括图像描述、视觉问答(VQA)和图像检索。图结构提供了一种自然的方式来表示图像中物体之间的关系布局,因此近年来,图神经网络(GNNs)已成为许多二维图像理解流程中的标准组件,尤其成为VQA任务组中的核心架构组件。本综述回顾了这一快速发展的领域,提供了二维图像理解方法中使用的图类型的分类体系、该领域使用的GNN模型全面清单,以及未来潜在发展的路线图。据我们所知,这是首篇涵盖以GNN为主要架构的图像描述、视觉问答和图像检索技术的全面综述。