Test suite minimization (TSM) is typically used to improve the efficiency of software testing by removing redundant test cases, thus reducing testing time and resources, while maintaining the fault detection capability of the test suite. Though many TSM approaches exist, most of them rely on code coverage (white-box) or model-based features, which are not always available for test engineers. Recent TSM approaches that rely only on test code (black-box) have been proposed, such as ATM and FAST-R. Though ATM achieves a better trade-off between effectiveness and efficiency than FAST-R, it suffers from scalability issues for large software systems as its execution time increases rapidly with test suite size. To address scalability, we propose LTM, a scalable and black-box similarity-based TSM approach based on language models. To support similarity measurement, we investigated three different pre-trained language models: CodeBERT, GraphCodeBERT, and UniXcoder, to extract embeddings of test code (Java test methods), on which we computed two similarity measures: Cosine Similarity and Euclidean Distance. Our goal is to find similarity measures that are not only computationally more efficient but can also better guide a Genetic Algorithm (GA), which is used for minimizing test suites, thus reducing minimization time. Experimental results showed that the best configuration of LTM (using UniXcoder with Cosine similarity) outperformed the best two configurations of ATM by achieving significantly higher fault detection rates (0.84 versus 0.81, on average) and, more importantly, running much faster (26.73 minutes versus 72.75 minutes, on average) than ATM, in terms of both preparation time (up to two orders of magnitude faster) and minimization time (one order of magnitude faster).

翻译：摘要：测试套件最小化（TSM）通常用于通过移除冗余测试用例来提高软件测试效率，从而减少测试时间和资源，同时保持测试套件的故障检测能力。尽管存在多种TSM方法，但大多数依赖于代码覆盖率（白盒）或基于模型的特征，而这些特征对测试工程师而言并不总是可用。近年来，研究者提出了仅依赖测试代码（黑盒）的TSM方法，例如ATM和FAST-R。尽管ATM在有效性与效率之间取得了比FAST-R更优的权衡，但其执行时间随测试套件规模快速增长，导致大型软件系统的可扩展性问题。为解决扩展性挑战，我们提出LTM，一种基于语言模型的可扩展黑盒相似性TSM方法。为支持相似性度量，我们研究了三种预训练语言模型：CodeBERT、GraphCodeBERT和UniXcoder，用于提取测试代码（Java测试方法）的嵌入表示，并基于此计算两种相似性度量：余弦相似度与欧氏距离。我们的目标是找到不仅计算效率更高，还能更有效指导遗传算法（GA）的相似性度量——该遗传算法用于最小化测试套件，从而缩短最小化时间。实验结果表明，LTM的最佳配置（使用UniXcoder与余弦相似度）在故障检测率上显著优于ATM的最佳两种配置（平均0.84对比0.81），更重要的是运行速度更快（平均26.73分钟对比72.75分钟），其中准备时间最多快两个数量级，最小化时间快一个数量级。