This paper reports our work on building up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. This is a part of an effort in building a STT system to aid dyslexic students who have cognitive deficiency in writing skills but have no problem expressing their ideas through speech. For Cantonese speech recognition, the basic unit of acoustic models can either be the conventional Initial-Final (IF) syllables, or the Onset-Nucleus-Coda (ONC) syllables where finals are further split into nucleus and coda to reflect the intra-syllable variations in Cantonese. By using the Kaldi toolkit, our system is trained using the stochastic gradient descent optimization model with the aid of GPUs for the hybrid Deep Neural Network and Hidden Markov Model (DNN-HMM) with and without I-vector based speaker adaptive training technique. The input features of the same Gaussian Mixture Model with speaker adaptive training (GMM-SAT) to DNN are used in all cases. Experiments show that the ONC-based syllable acoustic modeling with I-vector based DNN-HMM achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812.

翻译：本文报告了我们基于音节声学模型构建粤语语音转文本（STT）系统的工作。这是构建STT系统以辅助读写技能存在认知缺陷但能通过语言无障碍表达思想的阅读障碍学生项目的一部分。在粤语语音识别中，声学模型的基本单元可以是传统的声韵母（IF）音节，也可采用将韵母进一步拆分为韵腹和韵尾的起首-核音-韵尾（ONC）音节，以反映粤语内部音节变化。通过使用Kaldi工具包，我们的系统采用随机梯度下降优化模型进行训练，并借助GPU实现混合深度神经网络与隐马尔可夫模型（DNN-HMM），同时实验了是否采用基于I-向量的说话人自适应训练技术。所有情况均使用经过说话人自适应训练（GMM-SAT）的高斯混合模型作为DNN的输入特征。实验表明，基于ONC音节的声学建模结合I-向量DNN-HMM取得了最佳性能，词错误率（WER）为9.66%，实时因子（RTF）为1.38812。