Large Language Models (LLMs) are transforming diverse fields and gaining increasing influence as human proxies. This development underscores the urgent need for evaluating value orientations and understanding of LLMs to ensure their responsible integration into public-facing applications. This work introduces ValueBench, the first comprehensive psychometric benchmark for evaluating value orientations and value understanding in LLMs. ValueBench collects data from 44 established psychometric inventories, encompassing 453 multifaceted value dimensions. We propose an evaluation pipeline grounded in realistic human-AI interactions to probe value orientations, along with novel tasks for evaluating value understanding in an open-ended value space. With extensive experiments conducted on six representative LLMs, we unveil their shared and distinctive value orientations and exhibit their ability to approximate expert conclusions in value-related extraction and generation tasks. ValueBench is openly accessible at https://github.com/Value4AI/ValueBench.

翻译：大语言模型（LLMs）正在变革众多领域，并作为人类代理获得日益增长的影响力。这一发展凸显了评估LLMs价值取向与价值理解的迫切需求，以确保其负责任地融入面向公众的应用。本工作提出了ValueBench，这是首个用于评估LLMs价值取向与价值理解的综合性心理测量基准。ValueBench收集了来自44个成熟心理测量量表的数据，涵盖453个多维价值维度。我们提出了一种基于真实人机交互的评估流程来探测价值取向，同时设计了新颖的任务以在开放价值空间中评估价值理解。通过对六个代表性LLMs进行大量实验，我们揭示了它们共有及独特的价值取向，并展示了其在价值相关抽取与生成任务中逼近专家结论的能力。ValueBench已在 https://github.com/Value4AI/ValueBench 公开访问。