Configuration tuning for large software systems is generally challenging due to the complex configuration space and expensive performance evaluation. Most existing approaches follow a two-phase process, first learning a regression-based performance prediction model on available samples and then searching for the configurations with satisfactory performance using the learned model. Such regression-based models often suffer from the scarcity of samples due to the enormous time and resources required to run a large software system with a specific configuration. Moreover, previous studies have shown that even a highly accurate regression-based model may fail to discern the relative merit between two configurations, whereas performance comparison is actually one fundamental strategy for configuration tuning. To address these issues, this paper proposes CM-CASL, a Comparison-based performance Modeling approach for software systems via Collaborative Active and Semisupervised Learning. CM-CASL learns a classification model that compares the performance of two given configurations, and enhances the samples through a collaborative labeling process by both human experts and classifiers using an integration of active and semisupervised learning. Experimental results demonstrate that CM-CASL outperforms two state-of-the-art performance modeling approaches in terms of both classification accuracy and rank accuracy, and thus provides a better performance model for the subsequent work of configuration tuning.

翻译：大型软件系统的配置调优通常因复杂的配置空间和昂贵的性能评估而颇具挑战性。现有方法大多遵循两阶段流程：首先基于可用样本学习回归式性能预测模型，随后利用该模型搜索具有满意性能的配置。由于以特定配置运行大型软件系统需要耗费大量时间和资源，此类回归模型常受样本稀缺问题的困扰。此外，以往研究表明，即便高精度的回归模型也可能无法辨别两个配置间的相对优劣，而性能比较恰是配置调优的基础策略之一。为解决上述问题，本文提出CM-CASL——一种基于协作式主动与半监督学习的软件系统比较性能建模方法。CM-CASL通过学习分类模型来比较两个给定配置的性能，并通过人工专家与分类器协作标注过程（融合主动学习与半监督学习）来扩充样本。实验结果表明，CM-CASL在分类精度与排序精度两方面均优于两种最先进的性能建模方法，从而为后续配置调优工作提供了更优的性能模型。