The paper proposes a framework that combines behavioral and computational experiments employing fictional prompts as a novel tool for investigating cultural artifacts and social biases in storytelling both by humans and generative AI. The study analyzes 250 stories authored by crowdworkers in June 2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging methods from narratology and inferential statistics. Both crowdworkers and large language models responded to identical prompts about creating and falling in love with an artificial human. The proposed experimental paradigm allows a direct and controlled comparison between human and LLM-generated storytelling. Responses to the Pygmalionesque prompts confirm the pervasive presence of the Pygmalion myth in the collective imaginary of both humans and large language models. All solicited narratives present a scientific or technological pursuit. The analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more progressive in terms of gender roles and sexuality than those written by humans. While AI narratives with default settings and no additional prompting can occasionally provide innovative plot twists, they offer less imaginative scenarios and rhetoric than human-authored texts. The proposed framework argues that fiction can be used as a window into human and AI-based collective imaginary and social dimensions.
翻译:本文提出一个结合行为实验与计算实验的框架,采用虚构性提示作为创新工具,用于探究人类与生成式人工智能在叙事中呈现的文化产物与社会偏见。研究通过融合叙事学与推断统计学方法,分析了2019年6月由众包工作者创作的250篇故事,以及2023年3月由GPT-3.5和GPT-4生成的80篇故事。众包工作者与大型语言模型均对关于创造人造人并与之相爱的相同提示作出回应。所提出的实验范式实现了人类与LLM生成叙事之间的直接可控比较。对皮格马利翁式提示的回应证实了该神话在人类与大型语言模型集体想象中普遍存在。所有征集到的叙事均呈现科学或技术追求。分析表明,GPT-3.5(特别是GPT-4)生成的叙事在性别角色与性取向方面比人类创作更具进步性。虽然采用默认设置且无额外提示的AI叙事偶尔能提供创新的情节转折,但其场景构建与修辞手法相较于人类创作文本缺乏想象力。本框架论证了虚构叙事可作为窥探人类与人工智能集体想象及社会维度的窗口。