In the rapidly evolving landscape of large language models (LLMs), most research has primarily viewed them as independent individuals, focusing on assessing their capabilities through standardized benchmarks and enhancing their general intelligence. This perspective, however, tends to overlook the vital role of LLMs as user-centric services in human-AI collaboration. This gap in research becomes increasingly critical as LLMs become more integrated into people's everyday and professional interactions. This study addresses the important need to understand user satisfaction with LLMs by exploring four key aspects: comprehending user intents, scrutinizing user experiences, addressing major user concerns about current LLM services, and charting future research paths to bolster human-AI collaborations. Our study develops a taxonomy of 7 user intents in LLM interactions, grounded in analysis of real-world user interaction logs and human verification. Subsequently, we conduct a user survey to gauge their satisfaction with LLM services, encompassing usage frequency, experiences across intents, and predominant concerns. This survey, compiling 411 anonymous responses, uncovers 11 first-hand insights into the current state of user engagement with LLMs. Based on this empirical analysis, we pinpoint 6 future research directions prioritizing the user perspective in LLM developments. This user-centered approach is essential for crafting LLMs that are not just technologically advanced but also resonate with the intricate realities of human interactions and real-world applications.
翻译:在大型语言模型(LLMs)快速发展的背景下,大多数研究主要将它们视为独立个体,侧重于通过标准化基准评估其能力并提升其通用智能。然而,这种视角往往忽视了LLMs作为以用户为中心的服务在人机协作中的关键作用。随着LLMs日益融入人们的日常和专业交互,这一研究空白变得愈发重要。本研究通过探索四个关键方面来满足理解用户对LLMs满意度的迫切需求:理解用户意图、审视用户体验、解决用户对当前LLM服务的主要关切,以及绘制未来研究路径以加强人机协作。基于对真实用户交互日志和人工验证的分析,我们构建了LLM交互中7种用户意图的分类体系。随后,我们开展了一项用户调查,以评估其对LLM服务的满意度,涵盖使用频率、跨意图体验以及主要关切。该调查收集了411份匿名回复,揭示了关于用户与LLM交互现状的11项第一手见解。基于此项实证分析,我们确定了6个优先考虑用户视角的LLM发展未来研究方向。这种以用户为中心的方法对于打造既技术先进又契合人类交互和现实应用复杂现实的LLMs至关重要。