The integration of Large Language Models (LLMs) into various global cultures fundamentally presents a cultural challenge: LLMs must navigate interactions, respect social norms, and avoid transgressing cultural boundaries. However, it is still unclear if LLMs can adapt their outputs to diverse cultural norms. Our study focuses on this aspect. We introduce NormAd, a novel dataset, which includes 2.6k stories that represent social and cultural norms from 75 countries, to assess the ability of LLMs to adapt to different granular levels of socio-cultural contexts such as the country of origin, its associated cultural values, and prevalent social norms. Our study reveals that LLMs struggle with cultural reasoning across all contextual granularities, showing stronger adaptability to English-centric cultures over those from the Global South. Even with explicit social norms, the top-performing model, Mistral-7b-Instruct, achieves only 81.8\% accuracy, lagging behind the 95.6\% achieved by humans. Evaluation on NormAd further reveals that LLMs struggle to adapt to stories involving gift-giving across cultures. Due to inherent agreement or sycophancy biases, LLMs find it considerably easier to assess the social acceptability of stories that adhere to cultural norms than those that deviate from them. Our benchmark measures the cultural adaptability (or lack thereof) of LLMs, emphasizing the potential to make these technologies more equitable and useful for global audiences. We release the NormAd dataset and its associated code on GitHub.

翻译：将大语言模型（LLMs）整合到全球多元文化中本质上构成了一项文化挑战：LLMs必须驾驭交互过程、尊重社会规范并避免跨越文化边界。然而，LLMs能否使其输出适应不同的文化规范仍不明确。本研究聚焦于此问题。我们提出了NormAd——一个包含代表75个国家社会文化规范的2.6k个故事的新型数据集，用于评估LLMs适应不同粒度层级社会文化语境（包括原籍国、相关文化价值观及主流社会规范）的能力。研究发现，LLMs在所有语境粒度上都存在文化推理困难，对以英语为中心的文化表现出比对全球南方文化更强的适应性。即使在提供明确社会规范的情况下，表现最佳的模型Mistral-7b-Instruct也仅达到81.8%的准确率，远低于人类95.6%的水平。基于NormAd的评估进一步揭示，LLMs难以适应涉及跨文化赠礼的故事。由于固有的认同倾向或谄媚偏见，LLMs评估符合文化规范故事的社会可接受性，远比评估偏离规范的故事更为容易。本基准测试衡量了LLMs的文化适应性（及其缺失），强调了使这些技术更公平、更适用于全球受众的潜力。我们已在GitHub上发布NormAd数据集及相关代码。