Data sonification-mapping data variables to auditory variables, such as pitch or volume-is used for data accessibility, scientific exploration, and data-driven art (e.g., museum exhibitions) among others. While a substantial amount of research has been made on effective and intuitive sonification design, software support is not commensurate, limiting researchers from fully exploring its capabilities. We contribute Erie, a declarative grammar for data sonification, that enables abstractly expressing auditory mappings. Erie supports specifying extensible tone designs (e.g., periodic wave, sampling, frequency/amplitude modulation synthesizers), various encoding channels, auditory legends, and composition options like sequencing and overlaying. Using standard Web Audio and Web Speech APIs, we provide an Erie compiler for web environments. We demonstrate the expressiveness and feasibility of Erie by replicating research prototypes presented by prior work and provide a sonification design gallery. We discuss future steps to extend Erie toward other audio computing environments and support interactive data sonification.
翻译:数据声化——将数据变量映射到听觉变量(如音高或音量)——被用于数据无障碍访问、科学探索及数据驱动艺术(如博物馆展览)等多个领域。尽管在有效且直观的声化设计方面已有大量研究,但软件支持仍不匹配,限制了研究人员充分探索其能力。我们提出Erie——一种用于数据声化的声明式语法,能够抽象地表达听觉映射。Erie支持指定可扩展的音调设计(如周期波、采样、频率/幅度调制合成器)、多种编码通道、听觉图例以及序列化和叠加等组合选项。基于标准的Web Audio和Web Speech API,我们提供了适用于Web环境的Erie编译器。通过复现先前工作中的研究原型并构建声化设计图库,我们展示了Erie的表现力与可行性。最后,我们讨论了未来将Erie扩展到其他音频计算环境及支持交互式数据声化的方向。