Collaborative robots must effectively communicate their internal state to humans to enable a smooth interaction. Nonverbal communication is widely used to communicate information during human-robot interaction, however, such methods may also be misunderstood, leading to communication errors. In this work, we explore modulating the acoustic parameter values (pitch bend, beats per minute, beats per loop) of nonverbal auditory expressions to convey functional robot states (accomplished, progressing, stuck). We propose a reinforcement learning (RL) algorithm based on noisy human feedback to produce accurately interpreted nonverbal auditory expressions. The proposed approach was evaluated through a user study with 24 participants. The results demonstrate that: 1. Our proposed RL-based approach is able to learn suitable acoustic parameter values which improve the users' ability to correctly identify the state of the robot. 2. Algorithm initialization informed by previous user data can be used to significantly speed up the learning process. 3. The method used for algorithm initialization strongly influences whether participants converge to similar sounds for each robot state. 4. Modulation of pitch bend has the largest influence on user association between sounds and robotic states.

翻译：协作机器人必须有效向人类传达其内部状态，以实现顺畅交互。非言语交流在人机交互中广泛用于信息传递，但此类方法也可能被误解，导致通信错误。本研究探索通过调节非言语听觉表达的声音参数值（音高弯曲、每分钟节拍数、每循环节拍数）来传递机器人功能状态（已完成、进行中、卡住）。我们提出一种基于含噪人类反馈的强化学习算法，以生成能被准确解读的非言语听觉表达。通过包含24名参与者的用户研究对所提方法进行评估。结果表明：1. 我们提出的基于强化学习的方法能够学习适当的声学参数值，从而提升用户正确识别机器人状态的能力；2. 利用先前用户数据初始化算法可显著加速学习过程；3. 算法初始化方法对参与者是否就各机器人状态收敛至相似声音具有强烈影响；4. 音高弯曲的调节对用户将声音与机器人状态建立关联的影响最大。