We introduce a constraint-selection-based experiment design for measuring narrative preferences of Large Language Models (LLMs). This design offers an interpretable lens on LLMs' narrative selection behavior. We developed a library of 200 narratology-grounded constraints and prompted selections from six LLMs under three different instruction types: basic, quality-focused, and creativity-focused. Findings demonstrate that models consistently prioritize Style over narrative content elements like Event, Character, and Setting. Style preferences remain stable across models and instruction types, whereas content elements show cross-model divergence and instructional sensitivity. These results suggest that LLMs have latent narrative preferences, which should inform how the NLP community evaluates and deploys models in creative domains.
翻译:我们提出了一种基于约束选择的实验设计,用于测量大型语言模型(LLMs)的叙事偏好。该设计为理解LLM的叙事选择行为提供了可解释的视角。我们开发了一个包含200条叙事学约束的库,并在三种不同指令类型(基础型、质量导向型、创造力导向型)下,引导六个LLM进行选择。实验结果表明,模型始终优先考虑“风格”而非“事件”“角色”“背景”等叙事内容元素。风格偏好在不同模型和指令类型间保持稳定,而内容元素则表现出跨模型差异和指令敏感性。这些发现表明LLM存在潜在的叙事偏好,这应指导自然语言处理社区在创意领域中对模型进行评估与部署。