Automatic recognition of disordered speech remains a highly challenging task to date. The underlying neuro-motor conditions, often compounded with co-occurring physical disabilities, lead to the difficulty in collecting large quantities of impaired speech required for ASR system development. This paper presents novel variational auto-encoder generative adversarial network (VAE-GAN) based personalized disordered speech augmentation approaches that simultaneously learn to encode, generate and discriminate synthesized impaired speech. Separate latent features are derived to learn dysarthric speech characteristics and phoneme context representations. Self-supervised pre-trained Wav2vec 2.0 embedding features are also incorporated. Experiments conducted on the UASpeech corpus suggest the proposed adversarial data augmentation approach consistently outperformed the baseline speed perturbation and non-VAE GAN augmentation methods with trained hybrid TDNN and End-to-end Conformer systems. After LHUC speaker adaptation, the best system using VAE-GAN based augmentation produced an overall WER of 27.78% on the UASpeech test set of 16 dysarthric speakers, and the lowest published WER of 57.31% on the subset of speakers with "Very Low" intelligibility.

翻译：无序语音的自动识别至今仍是一项极具挑战性的任务。潜在的神经运动障碍（常伴随共存的身体残疾）导致难以收集开发ASR系统所需的大规模受损语音数据。本文提出基于变分自编码器生成对抗网络（VAE-GAN）的个性化无序语音增强方法，该方法能够同时学习编码、生成和区分合成的受损语音。通过推导分离的潜在特征，分别表征构音障碍语音特性与音素上下文表征。此外，还融入了自监督预训练的Wav2vec 2.0嵌入特征。在UASpeech语料库上进行的实验表明，本文提出的对抗性数据增强方法在混合TDNN和端到端Conformer系统中始终优于基线速度扰动及非VAE-GAN增强方法。经过LHUC说话人自适应后，基于VAE-GAN增强的最佳系统对UASpeech测试集中16位构音障碍说话人的总体词错误率（WER）为27.78%，在极低可懂度说话人子集上取得了已发表的最低WER 57.31%。