Reinforcement learning has been increasingly applied in monitoring applications because of its ability to learn from previous experiences and can make adaptive decisions. However, existing machine learning-based health monitoring applications are mostly supervised learning algorithms, trained on labels and they cannot make adaptive decisions in an uncertain complex environment. This study proposes a novel and generic system, predictive deep reinforcement learning (PDRL) with multiple RL agents in a time series forecasting environment. The proposed generic framework accommodates virtual Deep Q Network (DQN) agents to monitor predicted future states of a complex environment with a well-defined reward policy so that the agent learns existing knowledge while maximizing their rewards. In the evaluation process of the proposed framework, three DRL agents were deployed to monitor a subject's future heart rate, respiration, and temperature predicted using a BiLSTM model. With each iteration, the three agents were able to learn the associated patterns and their cumulative rewards gradually increased. It outperformed the baseline models for all three monitoring agents. The proposed PDRL framework is able to achieve state-of-the-art performance in the time series forecasting process. The proposed DRL agents and deep learning model in the PDRL framework are customized to implement the transfer learning in other forecasting applications like traffic and weather and monitor their states. The PDRL framework is able to learn the future states of the traffic and weather forecasting and the cumulative rewards are gradually increasing over each episode.

翻译：强化学习因其从过往经验中学习并做出自适应决策的能力，在监控应用中得到了日益广泛的应用。然而，现有的基于机器学习的健康监控应用大多为监督学习算法，依赖于标签训练，无法在不确定的复杂环境中做出自适应决策。本研究提出了一种新颖且通用的系统——预测性深度强化学习（PDRL），其在时间序列预测环境中部署多个强化学习（RL）智能体。所提出的通用框架容纳了虚拟深度Q网络（DQN）智能体，通过定义良好的奖励策略来监控复杂环境的预测未来状态，使智能体在学习现有知识的同时最大化其奖励。在评估该框架的过程中，我们部署了三个深度强化学习（DRL）智能体，用于监控通过双向长短期记忆（BiLSTM）模型预测的受试者未来心率、呼吸和体温。随着每次迭代，这三个智能体能够学习相关模式，其累积奖励逐步增加。在全部三个监控智能体上，其表现均优于基线模型。所提出的PDRL框架能够在时间序列预测过程中实现最先进的性能。PDRL框架中的DRL智能体和深度学习模型可定制化，用于在其他预测应用中（如交通和天气及其状态监控）实现迁移学习。该PDRL框架能够学习交通和天气预测的未来状态，且累积奖励随每个回合逐步增加。