Storytelling is a fundamental aspect of human communication, relying heavily on creativity to produce narratives that are novel, appropriate, and surprising. While large language models (LLMs) have recently demonstrated the ability to generate high-quality stories, their creative capabilities remain underexplored. Previous research has either focused on creativity tests requiring short responses or primarily compared model performance in story generation to that of professional writers. However, the question of whether LLMs exhibit creativity in writing short stories on par with the average human remains unanswered. In this work, we conduct a systematic analysis of creativity in short story generation across LLMs and everyday people. Using a five-sentence creative story task, commonly employed in psychology to assess human creativity, we automatically evaluate model- and human-generated stories across several dimensions of creativity, including novelty, surprise, and diversity. Our findings reveal that while LLMs can generate stylistically complex stories, they tend to fall short in terms of creativity when compared to average human writers.

翻译：讲故事是人类交流的基本方式，其高度依赖创造力来产生新颖、恰当且令人惊喜的叙事。尽管大型语言模型（LLMs）近期已展现出生成高质量故事的能力，但其创造性潜力仍未得到充分探索。先前研究要么聚焦于需要简短回答的创造力测试，要么主要将模型在故事生成方面的表现与专业作家进行比较。然而，大型语言模型在创作短篇故事时是否展现出与普通人相当的创造力，这一问题尚未得到解答。本研究对大型语言模型与普通人在短篇故事生成中的创造力进行了系统性分析。我们采用心理学中常用于评估人类创造力的五句子创意故事任务，从新颖性、惊喜度和多样性等多个创造力维度，自动评估模型生成与人类创作的故事。研究结果表明，尽管大型语言模型能够生成文体复杂的故事，但在创造力方面往往不及普通人类作者。