LLMs have been showing limitations when it comes to cultural coverage and competence, and in some cases show regional biases such as amplifying Western and Anglocentric viewpoints. While there have been works analysing the cultural capabilities of LLMs, there has not been specific work on highlighting LLM regional preferences when it comes to cultural-related questions. In this work, we propose a new dataset based on a comprehensive taxonomy of Culture-Related Open Questions (CROQ). The results show that, contrary to previous cultural bias work, LLMs show a clear tendency towards countries such as Japan. Moveover, our results show that when prompting in languages such as English or other high-resource ones, LLMs tend to provide more diverse outputs and show less inclinations towards answering questions highlighting countries for which the input language is an official language. Finally, we also investigate at which point of LLM training this cultural bias emerges, with our results suggesting that the first clear signs appear after supervised fine-tuning, and not during pre-training.

翻译：大语言模型在文化覆盖与能力方面已显露出局限性，某些情况下更呈现地域偏见，例如放大西方与盎格鲁中心主义视角。尽管已有研究分析大语言模型的文化能力，但尚未有专门工作聚焦其在文化相关提问中的地域偏好。本研究基于文化相关开放问题（CROQ）的综合分类体系提出新数据集。结果表明，与以往文化偏见研究结论相悖，大语言模型明显偏好日本等国家。此外，当使用英语或其他高资源语言提示时，模型输出更具多样性，且对输入语言为官方语言的国家相关问题的回答倾向性减弱。最后，本研究探究文化偏见在模型训练过程中的出现节点，结果显示首个明显信号出现在监督微调阶段，而非预训练阶段。