Although system heterogeneity has been extensively studied in the past, there is yet to be a study on measuring the impact of heterogeneity on system performance. For this purpose, we propose a heterogeneity measure that can characterize the impact of the heterogeneity of a system on its performance behavior in terms of throughput or makespan. We develop a mathematical model to characterize a heterogeneous system in terms of its task and machine heterogeneity dimensions and then reduce it to a single value, called Homogeneous Equivalent Execution Time (HEET), which represents the execution time behavior of the entire system. We used AWS EC2 instances to implement a real-world machine learning inference system. Performance evaluation of the HEET score across different heterogeneous system configurations demonstrates that HEET can accurately characterize the performance behavior of these systems. In particular, the results show that our proposed method is capable of predicting the true makespan of heterogeneous systems without online evaluations with an average precision of 84%. This heterogeneity measure is instrumental for solution architects to configure their systems proactively to be sufficiently heterogeneous to meet their desired performance objectives.

翻译：尽管系统异构性在过去已被广泛研究，但尚未有研究量化异构性对系统性能的影响。为此，我们提出一种异构性度量方法，能够表征系统异构性对其吞吐量或完工时间等性能行为的影响程度。我们构建了一个数学模型，从任务与机器两个异构性维度刻画异构系统，并将其降维为单一数值——同构等效执行时间（HEET），该数值表征整个系统的执行时间行为。通过使用AWS EC2实例搭建真实机器学习推理系统，对不同异构系统配置下的HEET分值进行性能评估，结果表明HEET能准确表征这些系统的性能行为。特别地，实验显示该方法无需在线评估即可预测异构系统的真实完工时间，平均精度达84%。该异构性度量有助于方案架构师主动配置系统，使其具备足够异构性以实现预期性能目标。