Test smells can pose difficulties during testing activities, such as poor maintainability, non-deterministic behavior, and incomplete verification. Existing research has extensively addressed test smells in automated software tests but little attention has been given to smells in natural language tests. While some research has identified and catalogued such smells, there is a lack of systematic approaches for their removal. Consequently, there is also a lack of tools to automatically identify and remove natural language test smells. This paper introduces a catalog of transformations designed to remove seven natural language test smells and a companion tool implemented using Natural Language Processing (NLP) techniques. Our work aims to enhance the quality and reliability of natural language tests during software development. The research employs a two-fold empirical strategy to evaluate its contributions. First, a survey involving 15 software testing professionals assesses the acceptance and usefulness of the catalog's transformations. Second, an empirical study evaluates our tool to remove natural language test smells by analyzing a sample of real-practice tests from the Ubuntu OS. The results indicate that software testing professionals find the transformations valuable. Additionally, the automated tool demonstrates a good level of precision, as evidenced by a F-Measure rate of 83.70%
翻译:测试坏味会在测试活动中引发困难,例如可维护性差、不确定性行为和验证不完整。现有研究已广泛探讨自动化软件测试中的测试坏味,但对自然语言测试中坏味的关注甚少。尽管已有研究识别并编录了此类坏味,但缺乏系统性的消除方法。因此,也缺乏自动识别和消除自然语言测试坏味的工具。本文介绍了一套旨在消除七种自然语言测试坏味的转换方法目录,以及一款基于自然语言处理技术实现的配套工具。我们的工作旨在提升软件开发过程中自然语言测试的质量与可靠性。本研究采用双管齐下的实证策略评估其贡献:首先,一项涉及15名软件测试专业人员的调查评估了该目录中转换方法的接受度与实用性;其次,通过分析来自Ubuntu操作系统的实际测试样本,开展了一项实证研究以评估我们工具消除自然语言测试坏味的效果。结果表明,软件测试专业人员认为这些转换方法具有价值。此外,该自动化工具展现出良好的精度,F-Measure值达到83.70%即是佐证。