Recently, machine and deep learning (ML/DL) algorithms have been increasingly adopted in many software systems. Due to their inductive nature, ensuring the quality of these systems remains a significant challenge for the research community. Unlike traditional software built deductively by writing explicit rules, ML/DL systems infer rules from training data. Recent research in ML/DL quality assurance has adapted concepts from traditional software testing, such as mutation testing, to improve reliability. However, it is unclear if these proposed testing techniques are adopted in practice, or if new testing strategies have emerged from real-world ML deployments. There is little empirical evidence about the testing strategies. To fill this gap, we perform the first fine-grained empirical study on ML testing in the wild to identify the ML properties being tested, the testing strategies, and their implementation throughout the ML workflow. We conducted a mixed-methods study to understand ML software testing practices. We analyzed test files and cases from 11 open-source ML/DL projects on GitHub. Using open coding, we manually examined the testing strategies, tested ML properties, and implemented testing methods to understand their practical application in building and releasing ML/DL software systems. Our findings reveal several key insights: 1.) The most common testing strategies, accounting for less than 40%, are Grey-box and White-box methods, such as Negative Testing, Oracle Approximation and Statistical Testing. 2.) A wide range of 17 ML properties are tested, out of which only 20% to 30% are frequently tested, including Consistency, Correctness}, and Efficiency. 3.) Bias and Fairness is more tested in Recommendation, while Security & Privacy is tested in Computer Vision (CV) systems, Application Platforms, and Natural Language Processing (NLP) systems.

翻译：近年来，机器学习和深度学习（ML/DL）算法在众多软件系统中得到日益广泛的应用。由于其归纳式特性，确保这类系统的质量仍是研究界面临的重大挑战。与通过编写显式规则进行演绎式构建的传统软件不同，ML/DL系统从训练数据中推断规则。近期关于ML/DL质量保证的研究借鉴了传统软件测试中的概念（如变异测试）以提升可靠性。然而，目前尚不清楚这些提出的测试技术是否在实践中被采纳，或者是否从现实世界的ML部署中涌现出新的测试策略。关于测试策略的实证证据仍然匮乏。为填补这一空白，我们首次对真实场景中的ML测试进行了细粒度实证研究，旨在识别被测试的ML属性、测试策略及其在整个ML工作流中的实现方式。我们采用混合方法研究以理解ML软件测试实践。我们分析了来自GitHub上11个开源ML/DL项目的测试文件与测试用例。通过开放式编码，我们手动检查了测试策略、被测试的ML属性以及已实现的测试方法，以理解它们在构建和发布ML/DL软件系统中的实际应用。我们的研究发现揭示了若干关键见解：1.) 最常见的测试策略（占比不足40%）是灰盒与白盒方法，例如负向测试、预言近似和统计测试。2.) 共有17种ML属性被测试，其中仅20%至30%被频繁测试，包括一致性、正确性和效率。3.) 偏差与公平性在推荐系统中得到更多测试，而安全与隐私性则在计算机视觉（CV）系统、应用平台以及自然语言处理（NLP）系统中被测试。