Achieving fairness in sequential-decision making systems within Human-in-the-Loop (HITL) environments is a critical concern, especially when multiple humans with different behavior and expectations are affected by the same adaptation decisions in the system. This human variability factor adds more complexity since policies deemed fair at one point in time may become discriminatory over time due to variations in human preferences resulting from inter- and intra-human variability. This paper addresses the fairness problem from an equity lens, considering human behavior variability, and the changes in human preferences over time. We propose FAIRO, a novel algorithm for fairness-aware sequential-decision making in HITL adaptation, which incorporates these notions into the decision-making process. In particular, FAIRO decomposes this complex fairness task into adaptive sub-tasks based on individual human preferences through leveraging the Options reinforcement learning framework. We design FAIRO to generalize to three types of HITL application setups that have the shared adaptation decision problem. Furthermore, we recognize that fairness-aware policies can sometimes conflict with the application's utility. To address this challenge, we provide a fairness-utility tradeoff in FAIRO, allowing system designers to balance the objectives of fairness and utility based on specific application requirements. Extensive evaluations of FAIRO on the three HITL applications demonstrate its generalizability and effectiveness in promoting fairness while accounting for human variability. On average, FAIRO can improve fairness compared with other methods across all three applications by 35.36%.

翻译：在人在回路环境中实现序贯决策系统的公平性是一个关键问题，特别是当多个具有不同行为和期望的人类受到系统中相同自适应决策的影响时。人类可变性因素增加了复杂性，因为由于个体间及个体内人类偏好的变化，某一时刻被认为公平的策略可能随时间推移而变得具有歧视性。本文从公平性视角出发，考虑人类行为可变性及人类偏好随时间的变化，致力于解决公平性问题。我们提出FAIRO——一种用于人在回路自适应中公平性感知序贯决策的新颖算法，将上述概念融入决策过程。具体而言，FAIRO通过利用Options强化学习框架，基于个体人类偏好将复杂公平性任务分解为自适应子任务。我们设计FAIRO以泛化至三类存在共享自适应决策问题的人在回路应用场景。此外，我们认识到公平性感知策略有时会与应用效用相冲突。为应对这一挑战，我们在FAIRO中提供公平性-效用权衡机制，使系统设计者能够根据具体应用需求平衡公平性与效用目标。在三类人在回路应用上的广泛评估表明，FAIRO在兼顾人类可变性同时促进公平性方面具有泛化性与有效性。平均而言，与所有三类应用中的其他方法相比，FAIRO可将公平性提升35.36%。