Performance modeling can help to improve the resource efficiency of clusters and distributed dataflow applications, yet the available modeling data is often limited. Collaborative approaches to performance modeling, characterized by the sharing of performance data or models, have been shown to improve resource efficiency, but there has been little focus on actual data sharing strategies and implementation in production environments. This missing building block holds back the realization of proposed collaborative solutions. In this paper, we envision, design, and evaluate a peer-to-peer performance data sharing approach for collaborative performance modeling of distributed dataflow applications. Our proposed data distribution layer enables access to performance data in a decentralized manner, thereby facilitating collaborative modeling approaches and allowing for improved prediction capabilities and hence increased resource efficiency. In our evaluation, we assess our approach with regard to deployment, data replication, and data validation, through experiments with a prototype implementation and simulation, demonstrating feasibility and allowing discussion of potential limitations and next steps.
翻译:性能建模有助于提升集群和分布式数据流应用的资源效率,然而可用的建模数据往往有限。已有研究表明,以性能数据或模型共享为特征的协同性能建模方法能够改善资源效率,但在实际生产环境中,针对具体的数据共享策略及其实施方案的研究仍较为薄弱。这一缺失环节阻碍了所提出的协同解决方案的落地实施。本文针对分布式数据流应用的协同性能建模,设计、评估并展望了一种对等性能数据共享方法。我们提出的数据分发层能够以去中心化方式实现对性能数据的访问,从而支持协同建模方法,提升预测能力并进而提高资源效率。在评估中,我们通过原型实现实验与仿真,从部署、数据复制及数据验证三个维度对所提方法进行验证,证明了其可行性,并探讨了潜在局限性与后续研究方向。