Reinforcement Learning (RL) has increasingly become a preferred method over traditional rule-based systems in diverse human-in-the-loop (HITL) applications due to its adaptability to the dynamic nature of human interactions. However, integrating RL in such settings raises significant privacy concerns, as it might inadvertently expose sensitive user information. Addressing this, our paper focuses on developing PAPER-HILT, an innovative, adaptive RL strategy through exploiting an early-exit approach designed explicitly for privacy preservation in HITL environments. This approach dynamically adjusts the tradeoff between privacy protection and system utility, tailoring its operation to individual behavioral patterns and preferences. We mainly highlight the challenge of dealing with the variable and evolving nature of human behavior, which renders static privacy models ineffective. PAPER-HILT's effectiveness is evaluated through its application in two distinct contexts: Smart Home environments and Virtual Reality (VR) Smart Classrooms. The empirical results demonstrate PAPER-HILT's capability to provide a personalized equilibrium between user privacy and application utility, adapting effectively to individual user needs and preferences. On average for both experiments, utility (performance) drops by 24%, and privacy (state prediction) improves by 31%.

翻译：强化学习（RL）在各类人机协同（HITL）应用中日益取代传统基于规则的系统，成为首选方法，这得益于其对人类交互动态特性的适应能力。然而，在此类场景中集成RL引发了显著的隐私问题，因为它可能无意中暴露敏感的用户信息。针对这一问题，本文聚焦于开发PAPER-HILT——一种创新的自适应RL策略，通过利用专为HITL环境隐私保护设计的早退方法实现。该方法动态调整隐私保护与系统效用之间的权衡，根据个体行为模式与偏好定制其运行方式。我们重点强调了处理人类行为多变且演化特性的挑战，这种特性使得静态隐私模型失效。通过在智能家居环境与虚拟现实（VR）智慧课堂两种不同场景中的应用，评估了PAPER-HILT的有效性。实证结果表明，PAPER-HILT能够实现用户隐私与应用效用之间的个性化均衡，有效适应个体用户需求与偏好。在两个实验中，平均效用（性能）下降24%，而隐私（状态预测）提升31%。