Controlling energy consumption in buildings through demand response (DR) has become increasingly important to reduce global carbon emissions and limit climate change. In this paper, we specifically focus on controlling the heating system of a residential building to optimize its energy consumption while respecting user's thermal comfort. Recent works in this area have mainly focused on either model-based control, e.g., model predictive control (MPC), or model-free reinforcement learning (RL) to implement practical DR algorithms. A specific RL method that recently has achieved impressive success in domains such as board games (go, chess) is Monte Carlo Tree Search (MCTS). Yet, for building control it has remained largely unexplored. Thus, we study MCTS specifically for building demand response. Its natural structure allows a flexible optimization that implicitly integrate exogenous constraints (as opposed, for example, to conventional RL solutions), making MCTS a promising candidate for DR control problems. We demonstrate how to improve MCTS control performance by incorporating a Physics-informed Neural Network (PiNN) model for its underlying thermal state prediction, as opposed to traditional purely data-driven Black-Box approaches. Our MCTS implementation aligned with a PiNN model is able to obtain a 3% increment of the obtained reward compared to a rule-based controller; leading to a 10% cost reduction and 35% reduction on temperature difference with the desired one when applied to an artificial price profile. We further implemented a Deep Learning layer into the Monte Carlo Tree Search technique using a neural network that leads the tree search through more optimal nodes. We then compared this addition with its Vanilla version, showing the improvement in computational cost required.

翻译：通过需求响应（DR）控制建筑能耗对于减少全球碳排放和限制气候变化已变得日益重要。本文重点研究住宅建筑供暖系统的控制，在尊重用户热舒适度的前提下优化其能耗。近年来该领域的研究主要聚焦于基于模型的控制方法（如模型预测控制，MPC）或无模型强化学习（RL）来实施实际DR算法。一种特定RL方法——蒙特卡洛树搜索（MCTS）近期在围棋、国际象棋等棋类博弈领域取得了显著成功，但在建筑控制领域仍鲜有探索。因此，本文专门研究MCTS在建筑需求响应中的应用。其天然结构支持灵活优化，能隐式整合外部约束（与传统RL解决方案形成对比），这使MCTS成为DR控制问题的理想候选方法。我们展示了如何通过引入物理信息神经网络（PiNN）模型进行底层热状态预测（区别于传统纯数据驱动黑箱方法）来提升MCTS控制性能。与基于规则的控制器相比，结合PiNN模型的MCTS实现可获得3%的奖励提升；应用于人工电价曲线时，实际成本降低10%，温度与目标值的偏差减少35%。我们进一步在蒙特卡洛树搜索技术中引入深度学习层，利用神经网络引导树搜索向更优节点探索，并将此增强版本与原始版本进行对比，展示了计算成本的改善效果。