2D image understanding is a complex problem within computer vision, but it holds the key to providing human-level scene comprehension. It goes further than identifying the objects in an image, and instead, it attempts to understand the scene. Solutions to this problem form the underpinning of a range of tasks, including image captioning, visual question answering (VQA), and image retrieval. Graphs provide a natural way to represent the relational arrangement between objects in an image, and thus, in recent years graph neural networks (GNNs) have become a standard component of many 2D image understanding pipelines, becoming a core architectural component, especially in the VQA group of tasks. In this survey, we review this rapidly evolving field and we provide a taxonomy of graph types used in 2D image understanding approaches, a comprehensive list of the GNN models used in this domain, and a roadmap of future potential developments. To the best of our knowledge, this is the first comprehensive survey that covers image captioning, visual question answering, and image retrieval techniques that focus on using GNNs as the main part of their architecture.
翻译:2D图像理解是计算机视觉中的一个复杂问题,但它关乎实现人类级别的场景理解能力。该问题不仅需要识别图像中的物体,更试图理解场景本身。对这一问题的解决方案构成了多项任务的基础,包括图像描述、视觉问答和图像检索。图结构能够自然地表达图像中物体之间的空间关系,因此近年来,图神经网络已成为众多2D图像理解流程的标准组件,尤其是在视觉问答类任务中成为核心架构。本综述系统梳理了这一快速发展的领域,提出了面向2D图像理解方法中使用的图类型分类体系,全面汇总了该领域应用的图神经网络模型,并展望了未来可能的研究方向。据我们所知,这是首篇系统涵盖图像描述、视觉问答和图像检索中图神经网络核心架构技术的综合性综述。