The rapid growth of audio-centric platforms and applications such as WhatsApp and Twitter has transformed the way people communicate and share audio content in modern society. However, these platforms are increasingly misused to disseminate harmful audio content, such as hate speech, deceptive advertisements, and explicit material, which can have significant negative consequences (e.g., detrimental effects on mental health). In response, researchers and practitioners have been actively developing and deploying audio content moderation tools to tackle this issue. Despite these efforts, malicious actors can bypass moderation systems by making subtle alterations to audio content, such as modifying pitch or inserting noise. Moreover, the effectiveness of modern audio moderation tools against such adversarial inputs remains insufficiently studied. To address these challenges, we propose MTAM, a Metamorphic Testing framework for Audio content Moderation software. Specifically, we conduct a pilot study on 2000 audio clips and define 14 metamorphic relations across two perturbation categories: Audio Features-Based and Heuristic perturbations. MTAM applies these metamorphic relations to toxic audio content to generate test cases that remain harmful while being more likely to evade detection. In our evaluation, we employ MTAM to test five commercial textual content moderation software and an academic model against three kinds of toxic content. The results show that MTAM achieves up to 38.6%, 18.3%, 35.1%, 16.7%, and 51.1% error finding rates (EFR) when testing commercial moderation software provided by Gladia, Assembly AI, Baidu, Nextdata, and Tencent, respectively, and it obtains up to 45.7% EFR when testing the state-of-the-art algorithms from the academy.

翻译：随着WhatsApp和Twitter等以音频为核心的平台和应用程序的快速发展，现代社会人们交流和分享音频内容的方式已发生深刻变革。然而，这些平台正日益被滥用于传播有害音频内容，如仇恨言论、欺诈性广告和露骨材料，这些内容可能产生严重的负面影响（例如对心理健康的有害影响）。为此，研究人员和从业者一直在积极开发和部署音频内容审核工具以应对此问题。尽管付出了这些努力，恶意行为者仍可通过细微修改音频内容（如调整音高或插入噪声）来规避审核系统。此外，现代音频审核工具对此类对抗性输入的有效性仍未得到充分研究。为应对这些挑战，我们提出了MTAM——一种面向音频内容审核软件的蜕变测试框架。具体而言，我们对2000个音频片段进行了初步研究，并定义了两类扰动（基于音频特征的扰动与启发式扰动）下的14种蜕变关系。MTAM将这些蜕变关系应用于有害音频内容，以生成仍具危害性但更可能规避检测的测试用例。在评估中，我们使用MTAM对五款商业文本内容审核软件和一个学术模型进行了三类有害内容的测试。结果表明，在测试Gladia、Assembly AI、百度、Nextdata和腾讯提供的商业审核软件时，MTAM分别实现了最高38.6%、18.3%、35.1%、16.7%和51.1%的错误发现率；在测试学术界最先进算法时，其错误发现率最高可达45.7%。