Internet of Things (IoT) devices are available in a multitude of scenarios, and provide constant, contextual data which can be leveraged to automatically reconfigure and optimize smart environments. To realize this vision, Artificial Intelligence (AI) and deep learning techniques are usually employed, however they need large quantity of data which is often not feasible in IoT scenarios. Digital Twins (DTs) have recently emerged as an effective way to replicate physical entities in the digital domain, to allow for simulation and testing of models and services. In this paper, we present a novel architecture based on the emerging Web of Things (WoT) standard, which provides a DT of a smart environment and applies Deep Reinforcement Learning (DRL) techniques on real time data. We implement our system in a real deployment, and test it along with a legacy system. Our findings show that the benefits of having a digital twin, specifically for DRL models, allow for faster convergence and finer tuning.

翻译：物联网设备广泛存在于各类场景中，可提供持续且具有情境感知的数据，这些数据能够被用于自动重新配置并优化智能环境。为实现这一目标，通常采用人工智能（AI）与深度学习技术，然而这类技术需要海量数据，这在物联网场景中往往难以实现。数字孪生（DT）近期被证明是一种有效的方法，可在数字域中复现物理实体，从而支持模型与服务的仿真测试。本文提出一种基于新兴Web of Things（WoT）标准的创新架构：该架构为智能环境构建数字孪生，并在实时数据上应用深度强化学习（DRL）技术。我们在实际部署中实现该系统，并与传统系统进行对比测试。研究结果表明，数字孪生（尤其对DRL模型而言）的优势在于能够实现更快的收敛速度与更精细的参数调优。