As AI becomes more prevalent throughout society, effective methods of integrating humans and AI systems that leverage their respective strengths and mitigate risk have become an important priority. In this paper, we introduce the paradigm of super reinforcement learning that takes advantage of Human-AI interaction for data driven sequential decision making. This approach utilizes the observed action, either from AI or humans, as input for achieving a stronger oracle in policy learning for the decision maker (humans or AI). In the decision process with unmeasured confounding, the actions taken by past agents can offer valuable insights into undisclosed information. By including this information for the policy search in a novel and legitimate manner, the proposed super reinforcement learning will yield a super-policy that is guaranteed to outperform both the standard optimal policy and the behavior one (e.g., past agents' actions). We call this stronger oracle a blessing from human-AI interaction. Furthermore, to address the issue of unmeasured confounding in finding super-policies using the batch data, a number of nonparametric and causal identifications are established. Building upon on these novel identification results, we develop several super-policy learning algorithms and systematically study their theoretical properties such as finite-sample regret guarantee. Finally, we illustrate the effectiveness of our proposal through extensive simulations and real-world applications.

翻译：随着人工智能在社会中的日益普及，如何有效整合人类与AI系统以发挥各自优势并降低风险已成为重要议题。本文提出一种利用人机交互进行数据驱动序贯决策的超级强化学习范式。该方法将观测到的行为（无论是AI还是人类行为）作为输入，为决策者（人类或AI）的策略学习提供更优的预言机。在存在未测量混杂因素的决策过程中，历史代理的行为可提供关于隐藏信息的宝贵线索。通过新颖且严谨的方式将这些信息纳入策略搜索，所提出的超级强化学习将生成一种超级策略，该策略被保证同时优于标准最优策略和行为策略（例如历史代理的行为）。我们将这种更强的预言机称为人机交互的福音。此外，为解决利用批数据寻找超级策略时存在的未测量混杂问题，本文建立了若干非参数化与因果识别方法。基于这些新的识别结果，我们开发了多种超级策略学习算法，并系统研究了其理论性质（如有限样本遗憾保证）。最后，通过大量仿真实验和实际应用案例验证了本方法的有效性。