The Internet of Things (IoT) is experiencing rapid growth. Smart devices are emerging in smart homes and industrial applications, performing mission-critical tasks. Bugs in IoT software can lead to severe consequences. For example, a buggy smart lock can allow unauthorized access to a private property. Testing is a primary practice to expose software bugs and ensure software quality. However, little is known about how IoT software is tested. To bridge this gap, we conducted the first empirical study on test cases in open-source IoT software. Specifically, we evaluated the effectiveness of test cases in IoT software, explored the challenges inherent in testing IoT software, and analyzed the usage of mock objects. Our results indicate that while IoT software often contains a considerable number of tests, their effectiveness remains limited. We identified the primary challenges in testing IoT software as managing complex interactions with various external dependencies, such as other network-reliant IoT components, file systems, operating systems, and databases. We also observed that the use of mock objects in IoT software closely aligns with our identified testing challenges. This alignment demonstrates the potential of mocking as a solution to enhance test coverage and address the complexities of IoT software testing.

翻译：物联网（IoT）正经历快速发展。智能设备在智能家居和工业应用中不断涌现，执行关键任务。物联网软件中的缺陷可能导致严重后果。例如，存在漏洞的智能锁可能允许未经授权的人员进入私人住宅。测试是暴露软件缺陷、保障软件质量的主要手段。然而，目前对物联网软件的测试方式知之甚少。为填补这一空白，我们对开源物联网软件中的测试用例开展了首次实证研究。具体而言，我们评估了物联网软件测试用例的有效性，探究了物联网软件测试中的固有挑战，并分析了模拟对象的使用情况。研究结果表明，尽管物联网软件通常包含大量测试，但其有效性仍有限。我们将物联网软件测试的主要挑战归纳为：管理与多种外部依赖项（如其他依赖网络的物联网组件、文件系统、操作系统及数据库）之间的复杂交互。我们还观察到，物联网软件中模拟对象的使用与我们识别的测试挑战高度契合。这种契合性表明，模拟技术有望成为提升测试覆盖率、应对物联网软件测试复杂性的有效方案。