Scene graph generation has emerged as a prominent research field in computer vision, witnessing significant advancements in the recent years. However, despite these strides, precise and thorough definitions for the metrics used to evaluate scene graph generation models are lacking. In this paper, we address this gap in the literature by providing a review and precise definition of commonly used metrics in scene graph generation. Our comprehensive examination clarifies the underlying principles of these metrics and can serve as a reference or introduction to scene graph metrics. Furthermore, to facilitate the usage of these metrics, we introduce a standalone Python package called SGBench that efficiently implements all defined metrics, ensuring their accessibility to the research community. Additionally, we present a scene graph benchmarking web service, that enables researchers to compare scene graph generation methods and increase visibility of new methods in a central place. All of our code can be found at https://lorjul.github.io/sgbench/.

翻译：场景图生成已成为计算机视觉领域的重要研究方向，近年来取得了显著进展。然而，尽管取得了这些成就，但用于评估场景图生成模型的指标仍缺乏精确而全面的定义。在本文中，我们通过综述并精确定义场景图生成中常用的指标来填补这一文献空白。我们的全面梳理阐明了这些指标背后的基本原理，可作为场景图指标的参考或入门介绍。此外，为促进这些指标的使用，我们开发了一个名为SGBench的独立Python包，高效实现了所有定义的指标，确保研究社区能够便捷使用。同时，我们提供一个场景图基准测试网络服务，使研究人员能够比较场景图生成方法，并在集中平台提升新方法的可见度。所有代码均可在https://lorjul.github.io/sgbench/获取。