Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for their dedication to provide unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.

翻译：大型语言模型（LLMs）有潜力通过其生成的内容（即AI生成内容，AIGC）改变我们的生活和工作。为充分利用这一变革，我们需要理解LLMs的局限性。本文研究了七个代表性LLMs（包括ChatGPT和LLaMA）生成的AIGC中的偏见。我们从以提供无偏见新闻报道闻名的《纽约时报》和路透社收集新闻文章，随后使用每条新闻的标题作为提示词，驱动每个受检LLM生成新闻内容，并通过对比AIGC与原始新闻文章，评估LLM生成的AIGC中的性别和种族偏见。我们进一步通过向基于这些新闻标题构建的提示词中加入性别偏见信息，分析了每个LLM在偏见提示下的性别偏见。研究揭示，每个受检LLM生成的AIGC均表现出显著的性别和种族偏见。此外，各LLM生成的AIGC对女性和黑人群体存在明显的歧视。在这些LLMs中，ChatGPT生成的AIGC偏见水平最低，且ChatGPT是唯一能在接收到偏见提示时拒绝内容生成的模型。