Comprehensive evaluation is one of the basis of experimental science. In High-Performance Graph Processing, a thorough evaluation of contributions becomes more achievable by supporting common input formats over different frameworks. However, each framework creates its specific format, which may not support reading large-scale real-world graph datasets. This shows a demand for high-performance libraries capable of loading graphs to (i)~accelerate designing new graph algorithms, (ii)~to evaluate the contributions on a wide range of graph algorithms, and (iii)~to facilitate easy and fast comparison over different graph frameworks. To that end, we present ParaGrapher, a high-performance API and library for loading large-scale and compressed graphs. ParaGrapher supports different types of requests for accessing graphs in shared- and distributed-memory and out-of-core graph processing. We explain the design of ParaGrapher and present a performance model of graph decompression, which is used for evaluation of ParaGrapher over three storage types. Our evaluation shows that by decompressing compressed graphs in WebGraph format, ParaGrapher delivers up to 3.2 times speedup in loading and up to 5.2 times speedup in end-to-end execution in comparison to the binary and textual formats. ParaGrapher is available online on https://blogs.qub.ac.uk/DIPSA/ParaGrapher/.
翻译:全面评估是实验科学的基础之一。在高性能图处理中,通过在不同框架上支持通用输入格式,可以更全面地评估研究成果。然而,每个框架都构建了其特有格式,这可能不支持读取大规模真实世界图数据集。这凸显了对高性能库的需求:既要能够加载图以加速新图算法的设计,又要便于在多种图算法上评估研究成果,同时促进不同图框架之间的便捷快速比较。为此,我们提出ParaGrapher——一个用于加载大规模压缩图的高性能API与库。ParaGrapher支持共享内存、分布式内存及外核图处理中多种类型的图访问请求。我们阐述了ParaGrapher的设计,并提出了图解压缩的性能模型,用于评估该库在三种存储类型上的表现。实验表明,通过解压WebGraph格式的压缩图,与二进制和文本格式相比,ParaGrapher在加载阶段可实现最高3.2倍的加速,在端到端执行中可实现最高5.2倍的加速。ParaGrapher已在https://blogs.qub.ac.uk/DIPSA/ParaGrapher/上开源提供。