Data sonification-mapping data variables to auditory variables, such as pitch or volume-is used for data accessibility, scientific exploration, and data-driven art (e.g., museum exhibitions) among others. While a substantial amount of research has been made on effective and intuitive sonification design, software support is not commensurate, limiting researchers from fully exploring its capabilities. We contribute Erie, a declarative grammar for data sonification, that enables abstractly expressing auditory mappings. Erie supports specifying extensible tone designs (e.g., periodic wave, sampling, frequency/amplitude modulation synthesizers), various encoding channels, auditory legends, and composition options like sequencing and overlaying. Using standard Web Audio and Web Speech APIs, we provide an Erie compiler for web environments. We demonstrate the expressiveness and feasibility of Erie by replicating research prototypes presented by prior work and provide a sonification design gallery. We discuss future steps to extend Erie toward other audio computing environments and support interactive data sonification.
翻译:数据声化——将数据变量映射到听觉变量(如音高或音量)——被应用于数据无障碍访问、科学探索及数据驱动艺术(如博物馆展览)等领域。尽管已有大量研究关注有效且直观的声化设计,但软件支持仍显不足,限制了研究者对其潜能的充分探索。我们提出了Erie,一种用于数据声化的声明式语法,能够抽象化表达听觉映射。Erie支持指定可扩展的音调设计(如周期波、采样、频率/幅度调制合成器)、多种编码通道、听觉图例以及序列化和叠加等组合选项。通过标准Web Audio和Web Speech API,我们为网络环境提供了Erie编译器。我们通过复现已有研究中的原型来展示Erie的表现力和可行性,并构建了声化设计画廊。最后,我们讨论了将Erie扩展到其他音频计算环境及支持交互式数据声化的未来方向。