Recent studies have highlighted the presence of cultural biases in Large Language Models (LLMs), yet often lack a robust methodology to dissect these phenomena comprehensively. Our work aims to bridge this gap by delving into the Food domain, a universally relevant yet culturally diverse aspect of human life. We introduce FmLAMA, a multilingual dataset centered on food-related cultural facts and variations in food practices. We analyze LLMs across various architectures and configurations, evaluating their performance in both monolingual and multilingual settings. By leveraging templates in six different languages, we investigate how LLMs interact with language-specific and cultural knowledge. Our findings reveal that (1) LLMs demonstrate a pronounced bias towards food knowledge prevalent in the United States; (2) Incorporating relevant cultural context significantly improves LLMs' ability to access cultural knowledge; (3) The efficacy of LLMs in capturing cultural nuances is highly dependent on the interplay between the probing language, the specific model architecture, and the cultural context in question. This research underscores the complexity of integrating cultural understanding into LLMs and emphasizes the importance of culturally diverse datasets to mitigate biases and enhance model performance across different cultural domains.

翻译：近期研究揭示了大语言模型（LLMs）中存在文化偏见，但往往缺乏系统的方法来全面剖析这些现象。本研究旨在通过深入探讨食物领域来弥补这一空白——食物既是人类生活中具有普遍性又蕴含文化多样性的重要方面。我们提出了FmLAMA，一个专注于食物相关文化事实及饮食实践差异的多语言数据集。我们分析了不同架构与配置下的LLMs，评估其在单语言与多语言环境中的表现。通过利用六种语言的模板，我们探究了LLMs如何与特定语言及文化知识进行交互。研究发现：（1）LLMs对美国普遍存在的食物知识表现出显著偏向；（2）融入相关文化语境能显著提升LLMs获取文化知识的能力；（3）LLMs捕捉文化细微差别的能力高度依赖于探测语言、具体模型架构及所涉文化语境之间的交互作用。本研究揭示了将文化理解融入LLMs的复杂性，并强调了文化多样性数据集在减轻偏见、提升模型跨不同文化领域性能方面的重要性。