Guidance can support users during the exploration and analysis of complex data. Previous research focused on characterizing the theoretical aspects of guidance in visual analytics and implementing guidance in different scenarios. However, the evaluation of guidance-enhanced visual analytics solutions remains an open research question. We tackle this question by introducing and validating a practical evaluation methodology for guidance in visual analytics. We identify eight quality criteria to be fulfilled and collect expert feedback on their validity. To facilitate actual evaluation studies, we derive two sets of heuristics. The first set targets heuristic evaluations conducted by expert evaluators. The second set facilitates end-user studies where participants actually use a guidance-enhanced system. By following such a dual approach, the different quality criteria of guidance can be examined from two different perspectives, enhancing the overall value of evaluation studies. To test the practical utility of our methodology, we employ it in two studies to gain insight into the quality of two guidance-enhanced visual analytics solutions, one being a work-in-progress research prototype, and the other being a publicly available visualization recommender system. Based on these two evaluations, we derive good practices for conducting evaluations of guidance in visual analytics and identify pitfalls to be avoided during such studies.

翻译：引导可以支持用户在探索和分析复杂数据时的操作。先前的研究侧重于描述视觉分析中引导的理论特征，并在不同场景中实现引导。然而，引导增强型视觉分析解决方案的评估仍是一个开放的研究问题。我们通过提出并验证一种实用的引导评估方法来应对这一问题。我们确定了八个需要满足的质量标准，并收集了专家对其有效性的反馈。为便于实际评估研究，我们推导出两组启发式规则。第一组针对由专家评估人员进行的启发式评估。第二组便于进行最终用户研究，其中参与者实际使用引导增强系统。通过采用这种双重方法，可以从两个不同角度审视引导的不同质量标准，从而提升评估研究的整体价值。为测试我们方法的实用性，我们在两项研究中应用它，以深入了解两个引导增强型视觉分析解决方案的质量，其中一个为进行中的研究原型，另一个为公开可用的可视化推荐系统。基于这两项评估，我们总结了进行视觉分析中引导评估的良好实践，并指出了此类研究中需要避免的陷阱。