Reinforcement Learning from Human Feedback has recently achieved significant success in various fields, and its performance is highly related to feedback quality. While much prior work acknowledged that human teachers' characteristics would affect human feedback patterns, there is little work that has closely investigated the actual effects. In this work, we designed an exploratory study investigating how human feedback patterns are associated with human characteristics. We conducted a public space study with two long horizon tasks and 46 participants. We found that feedback patterns are not only correlated with task statistics, such as rewards, but also correlated with participants' characteristics, especially robot experience and educational background. Additionally, we demonstrated that human feedback value can be more accurately predicted with human characteristics compared to only using task statistics. All human feedback and characteristics we collected, and codes for our data collection and predicting more accurate human feedback are available at https://github.com/AABL-Lab/CHARM

翻译：基于人类反馈的强化学习近期在多个领域取得了显著成功，其性能与反馈质量高度相关。尽管先前许多研究已认识到人类教师的特征会影响其反馈模式，但针对实际影响的深入探究仍较为缺乏。本研究设计了一项探索性实验，旨在探究人类反馈模式与个体特征之间的关联。我们在公共空间开展了包含两项长时程任务、涉及46名参与者的实验。研究发现，反馈模式不仅与任务统计量（如奖励值）相关，还与参与者的特征——特别是机器人操作经验和教育背景——存在关联。此外，我们证明相较于仅使用任务统计量，结合人类特征能够更准确地预测人类反馈的价值。本研究收集的所有人类反馈数据与特征数据，以及用于数据采集和实现更精准人类反馈预测的代码，均已公开于 https://github.com/AABL-Lab/CHARM。