Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and contrast with behaviour in a Verbal Fluency Task. We use sentence embeddings to identify response categories and compute semantic similarities, which we use to generate jump profiles. Our results corroborate earlier work in humans reporting both persistent (deep search in few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, where both pathways lead to similar creativity scores. LLMs were found to be biased towards either persistent or flexible paths, that varied across tasks. Though LLMs as a population match human profiles, their relationship with creativity is different, where the more flexible models score higher on creativity. Our dataset and scripts are available on \href{https://github.com/surabhisnath/Creative_Process}{GitHub}.
翻译:大型语言模型表现出相当的创造性,在创造性任务上常与人类平均水平相当。然而,关于LLM创造性的研究仅集中于“产品”,对创造性“过程”的关注甚少。人类创造性的过程分析通常需要手工编码类别或利用反应时间,这些方法不适用于LLMs。我们提出了一种自动化方法,用于描述人类和LLMs在替代用途任务中探索语义空间的过程,并将其与词语流畅性任务中的行为进行对比。我们使用句子嵌入来识别响应类别并计算语义相似度,进而生成跳跃轮廓。我们的结果证实了早期对人类的研究,报告了创造性的持续性(在少数语义空间中深入搜索)和灵活性(跨多个语义空间广泛搜索)两种路径,且两种路径均能产生相似的创造性得分。研究发现LLMs倾向于持续性或灵活性路径,且这一倾向因任务而异。尽管LLM群体与人类轮廓相匹配,但它们与创造性的关系不同——更灵活的模型在创造性上得分更高。我们的数据集和脚本可在GitHub上获取(https://github.com/surabhisnath/Creative_Process)。