Text-to-Text Transfer Transformer (T5) has recently been considered for the Grapheme-to-Phoneme (G2P) transduction. As a follow-up, a tokenizer-free byte-level model based on T5 referred to as ByT5, recently gave promising results on word-level G2P conversion by representing each input character with its corresponding UTF-8 encoding. Although it is generally understood that sentence-level or paragraph-level G2P can improve usability in real-world applications as it is better suited to perform on heteronyms and linking sounds between words, we find that using ByT5 for these scenarios is nontrivial. Since ByT5 operates on the character level, it requires longer decoding steps, which deteriorates the performance due to the exposure bias commonly observed in auto-regressive generation models. This paper shows that the performance of sentence-level and paragraph-level G2P can be improved by mitigating such exposure bias using our proposed loss-based sampling method.

翻译：文本到文本迁移转换器（T5）近期被用于字素到音素（G2P）转换任务。作为其后续研究，一种基于T5的无分词器字节级模型（称为ByT5）通过将每个输入字符编码为对应的UTF-8字节表示，在词级G2P转换中取得了令人鼓舞的结果。尽管人们普遍认为句子级或段落级G2P能更好地处理同形异义词及词间连音现象，更适用于实际应用场景，但我们发现将ByT5应用于这些场景并非易事。由于ByT5在字符级别操作，需要更长的解码步骤，这导致自回归生成模型中常见的暴露偏差问题加剧，从而降低模型性能。本文证明，通过我们提出的基于损失的采样方法缓解这种暴露偏差，可以显著提升句子级和段落级G2P的转换效果。