Evaluations of digital library information systems are typically centred on users correctly, efficiently, and quickly performing predefined tasks. Additionally, users generally enjoy working with the evaluated system, and completed questionnaires show an interface's excellent user experience. However, such evaluations do not explicitly consider comparing or connecting user-specific information-seeking behaviour with digital library system capabilities and thus overlook actual user needs or further system requirements. We aim to close this gap by introducing the usage of formalisations of users' task conduction strategies to compare their information needs with the capabilities of such information systems. We observe users' strategies in scope of expert finding and paper search. We propose and investigate using the business process model notation to formalise task conduction strategies and the SchenQL digital library interface as an example system. We conduct interviews in a qualitative evaluation with 13 participants from various backgrounds from which we derive models. We discovered that the formalisations are suitable and helpful to mirror the strategies back to users and to compare users' ideal task conductions with capabilities of information systems. We conclude using formal models for qualitative digital library studies being a suitable mean to identify current limitations and depict users' task conduction strategies. Our published dataset containing the evaluation data can be reused to investigate other digital library systems' fit for depicting users' ideal task solutions.
翻译:数字图书馆信息系统的评估通常侧重于用户正确、高效且快速地完成预定义任务。此外,用户通常乐于使用被评估的系统,完成的问卷也显示出界面的卓越用户体验。然而,此类评估并未明确考虑将用户特定的信息搜寻行为与数字图书馆系统能力进行对比或关联,因此忽略了实际用户需求或进一步的系统要求。我们旨在通过引入用户任务执行策略的形式化建模来弥合这一差距,从而将用户的信息需求与信息系统的能力进行比较。我们观察用户在专家查找和论文搜索范围内的策略。我们提出并研究使用业务流程模型符号来形式化任务执行策略,并以SchenQL数字图书馆界面作为示例系统。我们在定性评估中通过与13名来自不同背景的参与者进行访谈,从中推导出模型。我们发现这些形式化方法适合且有助于将策略反馈给用户,并比较用户理想的任务执行方式与信息系统的能力。我们得出结论,将形式化模型用于定性数字图书馆研究是识别当前局限性并描绘用户任务执行策略的合适手段。我们发布的包含评估数据的数据集可被重复利用,以研究其他数字图书馆系统在描绘用户理想任务解决方案方面的匹配程度。