Systems for heating, ventilation and air-conditioning (HVAC) of buildings are traditionally controlled by a rule-based approach. In order to reduce the energy consumption and the environmental impact of HVAC systems more advanced control methods such as reinforcement learning are promising. Reinforcement learning (RL) strategies offer a good alternative, as user feedback can be integrated more easily and presence can also be incorporated. Moreover, multi-agent RL approaches scale well and can be generalized. In this paper, we propose a multi-agent RL framework based on existing work that learns reducing on one hand energy consumption by optimizing HVAC control and on the other hand user feedback by occupants about uncomfortable room temperatures. Second, we show how to reduce training time required for proper RL-agent-training by using parameter sharing between the multiple agents and apply different pretraining techniques. Results show that our framework is capable of reducing the energy by around 6% when controlling a complete building or 8% for a single room zone. The occupants complaints are acceptable or even better compared to a rule-based baseline. Additionally, our performance analysis show that the training time can be drastically reduced by using parameter sharing.

翻译：建筑供暖、通风与空调系统传统上采用基于规则的方法进行控制。为降低暖通空调系统的能耗与环境影响，强化学习等先进控制方法前景广阔。强化学习策略提供了良好的替代方案，因其能更便捷地整合用户反馈并融入 occupancy 信息。此外，多智能体强化学习方法具有良好的可扩展性与泛化能力。本文在已有工作基础上提出一种多智能体强化学习框架，一方面通过优化暖通空调控制降低能耗，另一方面减少 occupant 对室温不适的投诉。其次，我们展示了如何通过多智能体间的参数共享以及应用不同的预训练技术，来缩短智能体训练所需的时间。结果表明，我们的框架在控制整栋建筑时能将能耗降低约6%，在控制单个房间区域时可降低约8%。与基于规则的基线方法相比，occupant 投诉数量可接受甚至更少。此外，性能分析表明，参数共享可大幅缩短训练时间。