While Large Language Model (LLM)-based agents can be used to create highly engaging interactive applications through prompting personality traits and contextual data, effectively assessing their personalities has proven challenging. This novel interdisciplinary approach addresses this gap by combining agent development and linguistic analysis to assess the prompted personality of LLM-based agents in a poetry explanation task. We developed a novel, flexible question bank, informed by linguistic assessment criteria and human cognitive learning levels, offering a more comprehensive evaluation than current methods. By evaluating agent responses with natural language processing models, other LLMs, and human experts, our findings illustrate the limitations of purely deep learning solutions and emphasize the critical role of interdisciplinary design in agent development.

翻译：尽管基于大型语言模型（LLM）的智能体可通过提示个性特征与上下文数据构建高度互动的应用，其个性评估仍面临挑战。本研究采用新颖的跨学科方法，结合智能体开发与语言分析，在诗歌解读任务中评估基于LLM的智能体所提示的个性特征。我们基于语言评估标准与人类认知学习层级，开发了一套新颖且灵活的题库，相比现有方法提供了更全面的评估框架。通过使用自然语言处理模型、其他LLM及人类专家对智能体响应进行评估，研究结果揭示了纯深度学习解决方案的局限性，并强调了跨学科设计在智能体开发中的关键作用。