Instrumental genesis through interdisciplinary collaboration-reflections on the emergence of a visualisation framework for video annotation data XML This paper presents, discusses and reflects on the development of a visualization framework for the analysis of the temporal dynamics of audiovisual expressivity. The main focus lies on the instrumental genesis process (Rabardel 1995; Longchamp 2012)-a concept trying to express and analyze the co-evolution of instruments and the practices they make possible-underlying this development. It is described through the collaboration and communication processes between computer science scholars and humanities scholars in finding new ways of visualizing complex datasets for exploration and presentation in the realm of film-studies research. It draws on the outcome and concrete usage of the visualizations in publications and presentations of a research group, the AdAproject, that investigates the audiovisual rhetorics of affect in audiovisual media on the financial crisis (2007-). These film analyses are based on theoretical assumptions on the process of film-viewing, the relation of the viewer's perception and the temporally unfolding audiovisual images, and a methodical approach that draws on 'steps' in the research process such as segmentation, description and qualification, called eMAEX (Kappelhoff et al. 2011-2016) to reconstruct these experiential figurations (Bakels et al. 2020a, 2020b). The main focus of this paper is the process of iterative development of visualizations as interactive interfaces generated with the open-source software Advene, that were an integral part of the research process. In this regard, the timeline visualization is not only of interest for visual argumentation in (digital) humanities publications, but also for the creation of annotations as well as the exploration of this data. In the first part of the paper we describe this interdisciplinary collaboration as instrumental genesis on a general level-as an evolving and iterative process. In the second part we focus on the specific challenge of designing a visualization framework for the temporal dynamics of audiovisual aesthetics. Lastly we zoom out by reflecting on experiences and insights that might be of interest for the wider digital humanities community.

翻译：本文呈现、讨论并反思了一款用于分析视听表达时间动态的可视化框架的开发过程。核心关注点在于该开发背后的"工具创生"过程（Rabardel 1995；Longchamp 2012）——这一概念旨在表达并分析工具及其所促成实践的共同演化。这一过程通过计算机科学学者与人文学者之间的协作与沟通得以描述，双方共同探索为电影研究领域复杂数据集的可视化新路径，以支持探索与展示。研究基于AdA项目研究组在出版物和报告中对可视化的实际应用成果，该项目聚焦2007年金融危机背景下视听媒体中情感音像修辞的分析。这些电影分析建立在关于观影过程的理论假设、观众感知与时间性展开的视听影像之关系，以及一种称为eMAEX的方法路径（Kappelhoff等人2011-2016）之上，该方法通过"分段、描述和定性"等研究步骤重构体验性构型（Bakels等人2020a, 2020b）。本文重点阐释了基于开源软件Advene生成的可视化交互界面的迭代开发过程——这些界面已成为研究进程的核心组成部分。在此意义上，时间线可视化不仅对（数字）人文学科出版物的视觉论证具有价值，更对注释生成与数据探索至关重要。论文第一部分从一般层面将这种跨学科协作描述为工具创生——一种演化迭代过程；第二部分聚焦于设计视听美学时间动态可视化框架的具体挑战；最后通过反思对更广泛数字人文界可能具有参考价值的经验与洞见进行总结。