Unit testing is an important practice that helps ensure the quality of a software system by validating its behavior through a series of test cases. Core to these test cases are assertion statements, which enable software practitioners to validate the correctness of the system's behavior. To aid with understanding and troubleshooting test case failures, practitioners can include a message (i.e., assertion message) within the assertion statement. While prior studies have examined the frequency and structure of assertion messages by mining software repositories, they do not determine their types or purposes or how practitioners perceive the need for or the usage of various types of assertion messages. In this paper, we survey 138 professional software practitioners to gather insights into their experience and views regarding assertion messages. Our findings reveal that a majority of survey respondents find assertion messages valuable for troubleshooting failures, improving test understandability, and serving as documentation. However, not all respondents consistently include messages in their assertion methods. We also identified common considerations for constructing effective assertion messages, challenges in crafting them, maintenance techniques, and their integration into debugging processes. Our results contribute to the understanding of current practices and provide guidelines for authoring high-quality assertion messages, serving as a foundation for best practices and coding standards. Furthermore, the insights can guide the improvement of automated unit testing tools by incorporating checks for the presence and quality of assertion messages and providing real-time feedback to practitioners.
翻译:单元测试是一种重要的实践,通过一系列测试用例验证软件系统的行为,有助于确保其质量。这些测试用例的核心是断言语句,它使软件从业者能够验证系统行为的正确性。为了帮助理解和排查测试用例失败,从业者可以在断言语句中包含一条消息(即断言消息)。尽管先前的研究通过挖掘软件仓库考察了断言消息的出现频率和结构,但并未确定其类型或目的,也未探究从业者如何看待各类断言消息的需求和使用方式。本文通过对138名专业软件从业者进行问卷调查,收集了关于他们使用断言消息的经验和观点。我们的研究发现,大多数受访者认为断言消息对于排查故障、提高测试可理解性以及作为文档具有重要价值。然而,并非所有受访者都会在其断言方法中始终包含消息。我们还识别了构建有效断言消息的常见考量因素、编写过程中面临的挑战、维护技术及其在调试过程中的整合方式。本研究结果有助于理解当前实践,并为撰写高质量断言消息提供指导原则,为最佳实践和编码标准奠定基础。此外,这些见解可指导自动化单元测试工具的改进,例如增加对断言消息存在性及质量的检查,并为从业者提供实时反馈。