Evaluation is essential to understanding the value that digital creativity brings to people's experience, for example in terms of their enjoyment, creativity, and engagement. There is a substantial body of research on how to design and evaluate interactive arts and digital creativity applications. There is also extensive Human-Computer Interaction (HCI) literature on how to evaluate user interfaces and user experiences. However, it can be difficult for artists, practitioners, and researchers to navigate such a broad and disparate collection of materials when considering how to evaluate technology they create that is at the intersection of art and interaction. This chapter provides a guide to designing robust user studies of creative applications at the intersection of art, technology and interaction, which we refer to as Media and Arts Technology (MAT). We break MAT studies down into two main kinds: proof-of-concept and comparative studies. As MAT studies are exploratory in nature, their evaluation requires the collection and analysis of both qualitative data such as free text questionnaire responses, interviews, and observations, and also quantitative data such as questionnaires, number of interactions, and length of time spent interacting. This chapter draws on over 15 years of experience of designing and evaluating novel interactive systems to provide a concrete template on how to structure a study to evaluate MATs that is both rigorous and repeatable, and how to report study results that are publishable and accessible to a wide readership in art and science communities alike.
翻译:评估对于理解数字创造力为人们体验带来的价值至关重要,例如在愉悦感、创造力和参与度方面。关于如何设计与评估交互式艺术及数字创造力应用,已有大量研究成果。人机交互领域亦积累了丰富的评估用户界面与用户体验的文献。然而,对于艺术家、实践者和研究者而言,在考虑如何评估其创作中处于艺术与交互交叉点的技术时,面对如此广泛而零散的资料往往感到无所适从。本章提供了一份指南,用于设计针对艺术、技术与交互交叉领域创意应用的稳健用户研究,我们将此类研究称为媒体与艺术技术。我们将媒体与艺术技术研究分为两大类型:概念验证研究与比较研究。由于媒体与艺术技术研究本质上是探索性的,其评估需要同时收集与分析定性数据(如自由文本问卷回复、访谈和观察记录)和定量数据(如问卷、交互次数和交互时长)。本章基于逾15年设计评估新型交互系统的经验,提供了一份具体模板,阐明如何构建既严谨又可重复的媒体与艺术技术研究评估结构,以及如何报告既适合发表又能被艺术与科学领域广泛读者群理解的研究结果。