Our life is getting filled by Internet of Things (IoT) devices. These devices often rely on closed or poorly documented protocols, with unknown formats and semantics. Learning how to interact with such devices in an autonomous manner is the key for interoperability and automatic verification of their capabilities. In this paper, we propose RL-IoT, a system that explores how to automatically interact with possibly unknown IoT devices. We leverage reinforcement learning (RL) to recover the semantics of protocol messages and to take control of the device to reach a given goal, while minimizing the number of interactions. We assume to know only a database of possible IoT protocol messages, whose semantics are however unknown. RL-IoT exchanges messages with the target IoT device, learning those commands that are useful to reach the given goal. Our results show that RL-IoT is able to solve both simple and complex tasks. With properly tuned parameters, RL-IoT learns how to perform actions with the target device, a Yeelight smart bulb in our case study, completing non-trivial patterns with as few as 400 interactions. RL-IoT paves the road for automatic interactions with poorly documented IoT protocols, thus enabling interoperable systems.

翻译：我们的生活正被各种事物的互联网(IoT)装置所填满。这些装置往往依赖封闭或记录不全的协议,其格式和语义不明。学习如何以自主方式与这些装置互动是互操作性和自动验证其能力的关键。在本文中,我们提议RL-IoT,这是一个探索如何与可能未知的 IoT 装置自动互动的系统。我们利用强化学习(RL)来恢复协议信件的语义,控制设备以达到一个特定目标,同时尽量减少互动的次数。我们假设只知道一个可能的 IoT 协议信息的数据库,其语义无论如何未知。RL-IoT 与目标 IoT 设备交换信息,学习那些对达到既定目标有用的命令。我们的结果表明,RL-IoT 能够解决简单和复杂的任务。通过适当调整的参数,RL-IoT 学会如何与目标设备进行操作动作,我们案例研究中的Yeeelight 智能灯泡,完成非 tripvial 模式,只有400个这样的自动互动系统。