A visualization notation is a recurring pattern of symbols used to author specifications of visualizations, from data transformation to visual mapping. Programmatic notations use symbols defined by grammars or domain-specific languages (e.g., ggplot2, dplyr, Vega-Lite) or libraries (e.g., Matplotlib, Pandas). Designers and prospective users of grammars and libraries often evaluate visualization notations by inspecting galleries of examples. While such collections demonstrate usage and expressiveness, their construction and evaluation are usually ad hoc, making comparisons of different notations difficult. More rarely, experts analyze notations via usability heuristics, such as the Cognitive Dimensions of Notations framework. These analyses, akin to structured close readings of text, can reveal design deficiencies, but place a burden on the expert to simultaneously consider many facets of often complex systems. To alleviate these issues, we introduce a metrics-based approach to usability evaluation and comparison of notations in which metrics are computed for a gallery of examples across a suite of notations. While applicable to any visualization domain, we explore the utility of our approach via a case study considering statistical graphics that explores 40 visualizations across 9 widely used notations. We facilitate the computation of appropriate metrics and analysis via a new tool called NotaScope. We gathered feedback via interviews with authors or maintainers of prominent charting libraries (n=6). We find that this approach is a promising way to formalize, externalize, and extend evaluations and comparisons of visualization notations.

翻译：可视化符号是用于编写可视化规范（从数据变换到视觉映射）的符号重复模式。程序化符号使用由语法或领域特定语言（如ggplot2、dplyr、Vega-Lite）或库（如Matplotlib、Pandas）定义的符号。语法和库的设计者及潜在用户通常通过浏览示例集锦来评估可视化符号。虽然这类集合展示了用法和表达能力，但其构建和评估通常较为随意，导致不同符号间的比较困难。更少见的是，专家通过可用性启发式方法（如符号认知维度框架）分析符号。这些分析类似于对文本进行结构化精读，可揭示设计缺陷，但专家需同时考虑复杂系统多个方面，负担较重。为解决这些问题，我们提出基于度量的可用性评估方法，通过为一套符号体系中的示例集锦计算度量指标来实现符号比较。该方法适用于任何可视化领域，我们通过统计图形案例研究探索其实用性，该研究涉及9种广泛使用的符号体系中的40种可视化图表。我们开发了新工具NotaScope以促进适当度量的计算与分析。通过对主流图表库作者或维护者进行访谈（n=6）收集反馈发现，该方法有望形式化、外化和扩展可视化符号的评估与比较。