The Internet of Things connects a plethora of smart devices globally across various applications like smart cities, autonomous vehicles and health monitoring. Simulation plays a key role in the testing of IoT systems, noting that field testing of a complete IoT product may be infeasible or prohibitively expensive. This paper addresses a specific yet important need in simulation-based testing for IoT: Stress testing of cloud systems. Existing stress testing solutions for IoT demand significant computational resources, making them ill-suited and costly. We propose a lean simulation framework designed for IoT cloud stress testing which enables efficient simulation of a large array of IoT and edge devices that communicate with the cloud. To facilitate simulation construction for practitioners, we develop a domain-specific language (DSL), named IoTECS, for generating simulators from model-based specifications. We provide the syntax and semantics of IoTECS and implement IoTECS using Xtext and Xtend. We assess simulators generated from IoTECS specifications for stress testing two real-world systems: a cloud-based IoT monitoring system and an IoT-connected vehicle system. Our empirical results indicate that simulators created using IoTECS: (1)achieve best performance when configured with Docker containerization; (2)effectively assess the service capacity of our case-study systems, and (3)outperform industrial stress-testing baseline tools, JMeter and Locust, by a factor of 3.5 in terms of the number of IoT and edge devices they can simulate using identical hardware resources. To gain initial insights about the usefulness of IoTECS in practice, we interviewed two engineers from our industry partner who have firsthand experience with IoTECS. Feedback from these interviews suggests that IoTECS is effective in stress testing IoT cloud systems, saving significant time and effort.
翻译:物联网在全球范围内连接了众多智能设备,广泛应用于智慧城市、自动驾驶汽车和健康监测等领域。仿真在物联网系统测试中起着关键作用,因为对完整物联网产品进行现场测试可能不可行或成本过高。本文针对物联网仿真测试中一个具体而重要的需求:云系统压力测试。现有的物联网压力测试解决方案需要大量计算资源,使其适用性受限且成本高昂。我们提出了一种专为物联网云压力测试设计的轻量级仿真框架,能够高效仿真大量与云通信的物联网及边缘设备。为便于实践者构建仿真,我们开发了一种名为IoTECS的领域特定语言,用于从基于模型的规约生成仿真器。我们给出了IoTECS的语法和语义,并使用Xtext和Xtend实现了IoTECS。我们评估了由IoTECS规约生成的仿真器对两个实际系统的压力测试效果:基于云的物联网监控系统和物联网车联网系统。实证结果表明,使用IoTECS创建的仿真器:(1)在配置Docker容器化时达到最佳性能;(2)能有效评估案例研究系统的服务容量;(3)在相同硬件资源下,可仿真的物联网及边缘设备数量比工业压力测试基准工具JMeter和Locust高出3.5倍。为初步了解IoTECS在实际应用中的有效性,我们访谈了两位来自工业合作伙伴、具有IoTECS使用经验的工程师。访谈反馈表明,IoTECS能有效进行物联网云系统压力测试,并显著节省时间和精力。