Existing research primarily evaluates the values of LLMs by examining their stated inclinations towards specific values. However, the "Value-Action Gap," a phenomenon rooted in environmental and social psychology, reveals discrepancies between individuals' stated values and their actions in real-world contexts. To what extent do LLMs exhibit a similar gap between their stated values and their actions informed by those values? This study introduces ValueActionLens, an evaluation framework to assess the alignment between LLMs' stated values and their value-informed actions. The framework encompasses the generation of a dataset comprising 14.8k value-informed actions across twelve cultures and eleven social topics, and two tasks to evaluate how well LLMs' stated value inclinations and value-informed actions align across three different alignment measures. Extensive experiments reveal that the alignment between LLMs' stated values and actions is sub-optimal, varying significantly across scenarios and models. Analysis of misaligned results identifies potential harms from certain value-action gaps. To predict the value-action gaps, we also uncover that leveraging reasoned explanations improves performance. These findings underscore the risks of relying solely on the LLMs' stated values to predict their behaviors and emphasize the importance of context-aware evaluations of LLM values and value-action gaps.

翻译：现有研究主要通过考察大型语言模型对特定价值观的表述倾向来评估其价值观。然而，源于环境与社会心理学的“价值-行动差距”现象揭示了个人所宣称的价值观与其在现实情境中的实际行动之间存在差异。大型语言模型在多大程度上表现出类似的、其宣称的价值观与基于这些价值观的行动之间的差距？本研究提出ValueActionLens评估框架，用于评估大型语言模型宣称的价值观与其基于价值观的行动之间的一致性。该框架包含：生成一个涵盖十二种文化和十一个社会主题、包含14.8k个基于价值观行动的数据集；以及设计两项任务，通过三种不同的对齐度量来评估大型语言模型宣称的价值观倾向与基于价值观的行动之间的一致性程度。大量实验表明，大型语言模型宣称的价值观与其行动之间的一致性并不理想，且在不同场景和模型间差异显著。对不一致结果的分析揭示了某些价值-行动差距可能带来的潜在危害。为了预测价值-行动差距，我们还发现利用推理解释能提升预测性能。这些发现凸显了仅依赖大型语言模型宣称的价值观来预测其行为所存在的风险，并强调了结合具体情境评估大型语言模型价值观及其价值-行动差距的重要性。