Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for delivering relatively unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.

翻译：大型语言模型（LLMs）通过其生成的内容（即AI生成内容，AIGC）有望改变我们的生活和工作方式。为利用这一变革，我们需要理解LLMs的局限性。本文研究了包括ChatGPT和LLaMA在内的七个代表性LLM所生成AIGC的偏见。我们收集了《纽约时报》和路透社中公认相对无偏见的新闻文章。随后，以这些新闻文章的标题作为提示，让每个被考察的LLM生成新闻内容，并通过比较AIGC与原始新闻文章，评估LLM所生成AIGC中的性别和种族偏见。此外，我们通过在上述新闻标题构建的提示中加入带有性别偏见的消息，分析了每个LLM在偏见性提示下的性别偏见。本研究发现，每个被考察LLM所生成的AIGC均表现出显著的性别和种族偏见。同时，各LLM生成的AIGC对女性和黑人群体存在明显的歧视。在众多LLM中，ChatGPT生成的AIGC偏见水平最低，且ChatGPT是唯一能够在被提供偏见性提示时拒绝生成内容的模型。