Performance modeling can help to improve the resource efficiency of clusters and distributed dataflow applications, yet the available modeling data is often limited. Collaborative approaches to performance modeling, characterized by the sharing of performance data or models, have been shown to improve resource efficiency, but there has been little focus on actual data sharing strategies and implementation in production environments. This missing building block holds back the realization of proposed collaborative solutions. In this paper, we envision, design, and evaluate a peer-to-peer performance data sharing approach for collaborative performance modeling of distributed dataflow applications. Our proposed data distribution layer enables access to performance data in a decentralized manner, thereby facilitating collaborative modeling approaches and allowing for improved prediction capabilities and hence increased resource efficiency. In our evaluation, we assess our approach with regard to deployment, data replication, and data validation, through experiments with a prototype implementation and simulation, demonstrating feasibility and allowing discussion of potential limitations and next steps.
翻译:性能建模有助于提升集群和分布式数据流应用的资源效率,但可用的建模数据往往有限。以性能数据或模型共享为特征的协同式性能建模方法已被证实能够改善资源效率,然而现有研究对生产环境中实际数据共享策略及具体实现的关注甚少。这一缺失的关键环节阻碍了协同解决方案的实现。本文构想、设计并评估了一种面向分布式数据流应用协同性能建模的对等性能数据共享方法。我们提出的数据分发层能够以去中心化方式实现对性能数据的访问,从而支撑协同建模方法,提升预测能力并最终提高资源效率。在评估环节中,我们通过原型实现实验与仿真,从部署策略、数据复制及数据验证三个维度验证了该方法的可行性,并探讨了潜在局限性及后续研究方向。