Large language models (LLMs) are increasingly used in robotics, especially for high-level action planning. Meanwhile, many robotics applications involve human supervisors or collaborators. Hence, it is crucial for LLMs to generate socially acceptable actions that align with people's preferences and values. In this work, we test whether LLMs capture people's intuitions about behavior judgments and communication preferences in human-robot interaction (HRI) scenarios. For evaluation, we reproduce three HRI user studies, comparing the output of LLMs with that of real participants. We find that GPT-4 strongly outperforms other models, generating answers that correlate strongly with users' answers in two studies $\unicode{x2014}$ the first study dealing with selecting the most appropriate communicative act for a robot in various situations ($r_s$ = 0.82), and the second with judging the desirability, intentionality, and surprisingness of behavior ($r_s$ = 0.83). However, for the last study, testing whether people judge the behavior of robots and humans differently, no model achieves strong correlations. Moreover, we show that vision models fail to capture the essence of video stimuli and that LLMs tend to rate different communicative acts and behavior desirability higher than people.

翻译：大型语言模型（LLMs）越来越多地被用于机器人领域，特别是在高级动作规划方面。同时，许多机器人应用涉及人类监督者或协作者。因此，LLMs必须能够生成符合社会接受度、与人类偏好和价值观一致的行为。在本研究中，我们测试了LLMs是否能够捕捉人类在人机交互（HRI）场景中对行为判断和交流偏好的直觉。为进行评估，我们复现了三个HRI用户研究，将LLM的输出与真实参与者的回答进行比较。我们发现，GPT-4显著优于其他模型，在两个研究中其生成的回答与用户的回答高度相关——第一个研究涉及在多种情境下为机器人选择最合适的交流行为（$r_s$ = 0.82），第二个研究涉及判断行为的合意性、意向性和意外性（$r_s$ = 0.83）。然而，在最后一个研究（测试人们是否对机器人和人类的行为有不同判断）中，没有任何模型实现强相关性。此外，我们表明视觉模型未能捕捉视频刺激的本质，且与人类相比，LLMs往往对不同的交流行为和行为的合意性给出更高的评分。