Voice-enabled technology is quickly becoming ubiquitous, and is constituted from machine learning (ML)-enabled components such as speech recognition and voice activity detection. However, these systems don't yet work well for everyone. They exhibit bias - the systematic and unfair discrimination against individuals or cohorts of individuals in favour of others (Friedman & Nissembaum, 1996) - across axes such as age, gender and accent. ML is reliant on large datasets for training. Dataset documentation is designed to give ML Practitioners (MLPs) a better understanding of a dataset's characteristics. However, there is a lack of empirical research on voice dataset documentation specifically. Additionally, while MLPs are frequent participants in fairness research, little work focuses on those who work with voice data. Our work makes an empirical contribution to this gap. Here, we combine two methods to form an exploratory study. First, we undertake 13 semi-structured interviews, exploring multiple perspectives of voice dataset documentation practice. Using open and axial coding methods, we explore MLPs' practices through the lenses of roles and tradeoffs. Drawing from this work, we then purposively sample voice dataset documents (VDDs) for 9 voice datasets. Our findings then triangulate these two methods, using the lenses of MLP roles and trade-offs. We find that current VDD practices are inchoate, inadequate and incommensurate. The characteristics of voice datasets are codified in fragmented, disjoint ways that often do not meet the needs of MLPs. Moreover, they cannot be readily compared, presenting a barrier to practitioners' bias reduction efforts. We then discuss the implications of these findings for bias practices in voice data and speech technologies. We conclude by setting out a program of future work to address these findings -- that is, how we may "right the docs".

翻译：语音技术正迅速普及，其核心依赖于机器学习组件，如语音识别和语音活动检测。然而，这些系统尚未实现普适性，仍存在偏差——即系统性地、不公平地歧视某些个体或群体（Friedman & Nissembaum, 1996）——表现在年龄、性别和口音等多个维度。机器学习依赖大规模数据集进行训练。数据集文档旨在帮助机器学习从业者更深入地理解数据集特征。然而，目前缺乏针对语音数据集文档的实证研究。此外，尽管机器学习从业者常参与公平性研究，但针对语音数据处理者的研究甚少。本研究旨在填补这一实证空白。我们采用两种方法形成探索性研究：首先，通过13次半结构化访谈，从多视角探究语音数据集文档实践；运用开放式编码和轴心编码方法，以角色与权衡为分析框架，剖析从业者的实践行为。基于此，我们进一步有目的地选取9个语音数据集的相关文档作为样本。随后，通过三角互证法整合两种方法的研究发现，聚焦从业者角色与权衡视角。研究发现，当前语音数据集文档实践尚处萌芽阶段，存在不充分、不匹配等问题。语音数据集特征以碎片化、不连贯的方式编码，常无法满足从业者需求。此外，文档缺乏可比较性，阻碍了从业者减少偏差的努力。最后，我们探讨了这些发现对语音数据与语音技术中偏差实践的启示，并规划了未来研究方向——即如何“正确文档”。