Offload of MPI collectives to network devices, e.g., NICs and switches, is being implemented as an effective mechanism to improve application performance by reducing inter- and intra-node communication and bypassing MPI software layers. Given the rich deployment of accelerators and programmable NICs/switches in data centers, we posit that there is an opportunity to further improve performance by extending this idea (of in-network collective processing) to a new class of more complex collectives. The most basic type of complex collective is the fusion of existing collectives. In previous work we have demonstrated the efficacy of this additional hardware and software support and shown that it can substantially improve the performance of certain applications. In this work we extend this approach. We seek to characterize a large number of MPI applications to determine overall applicability, both breadth and type, and so provide insight for hardware designers and MPI developers about future offload possibilities. Besides increasing the scope of prior surveys to include finding (potential) new MPI constructs, we also tap into new methods to extend the survey process. Prior surveys on MPI usage considered lists of applications constructed based on application developers' knowledge. The approach taken in this paper, however, is based on an automated mining of a large collection of code sources. More specifically, the mining is accomplished by GitHub REST APIs. We use a database management system to store the results and to answer queries. Another advantage is that this approach provides support for a more complex analysis of MPI usage, which is accomplished by user queries.

翻译：将MPI集合操作卸载到网络设备（如网卡和交换机）上，已成为一种通过减少节点间及节点内通信、绕过MPI软件层来提升应用性能的有效机制。鉴于数据中心中加速器及可编程网卡/交换机的广泛部署，我们认为存在一个机遇：通过将这种网络内集合处理的思想扩展至一类更复杂的集合体，可进一步优化性能。其中最基础的复杂集合体类型是现有集合体的融合。在前期工作中，我们已验证了这种额外硬件与软件支持的有效性，并证明其能显著提升特定应用的性能。本文在此基础上进一步扩展研究方法，旨在通过大规模MPI应用的特征分析确定其整体适用性（包括广度与类型），从而为硬件设计者与MPI开发者提供未来卸载可行性的参考。除扩大先前调查范围以发现（潜在）新型MPI结构外，我们还引入新方法扩展调查流程。以往的MPI使用调查主要基于开发者知识构建的应用列表，而本文采用的方法基于对大量代码源的自动化挖掘。具体而言，该挖掘通过GitHub REST API实现。我们利用数据库管理系统存储结果并响应查询。此方法的另一优势在于，它能通过用户查询支持对MPI使用模式的更复杂分析。