Objective: laryngectomees depend on an electromechanical device to generate electrolaryngeal (EL) speech. Compared with normal speech, EL speech suffers from severe distortion, limited phonetic variation, unnatural prosody, and temporal shifts, degrading naturalness and intelligibility. Although sequence-to-sequence (seq2seq) voice conversion (VC) based EL-speech-to-normal-speech conversion (EL2SP) is promising, substantial mismatches between EL and normal speech inevitably cause cumulative mapping errors that limit performance. To address this, we describe a novel representation learning framework integrating speech and text representations to improve mapping and reconstruction quality within a seq2seq VC model. Methods: our methodology comprises two main stages: 1) representation integration and learning, and 2) reconstruction training. A network capable of incorporating auxiliary text information is first constructed with pretrained modules to learn speech--text-based integrated representations. Then, an autoencoder-style reconstruction strategy finalizes EL2SP model to inherit these representations without increasing model complexity. We introduce three fusion strategies including middle-, input-, and hybrid-level fusion strategies that progressively enhance learning. Moreover, besides standard seq2seq VC objectives, an additional reconstruction loss on the integrated representation is introduced to refine representation transfer. Results: experiments under different EL2SP datasets consistently demonstrate that our methods, combined with data augmentations, outperform baselines relying solely on speech representations. Furthermore, progressive improvements with system design depth validate the effectiveness of our methods. Significance: the proposed methods provide an extensible and practical methodology for EL speech enhancement and assistive communication technologies.

翻译：目的：喉切除患者依赖机电设备产生电喉语音。与正常语音相比，电喉语音存在严重失真、音素变化受限、韵律不自然及时间偏移等问题，导致自然度和可懂度下降。尽管基于序列到序列语音转换的电喉语音到正常语音转换方法具有潜力，但电喉语音与正常语音之间的显著差异不可避免地导致累积映射误差，从而限制性能。为解决此问题，我们提出一种新颖的表示学习框架，融合语音和文本表示，以提升序列到序列语音转换模型中的映射与重建质量。方法：本方法包含两个主要阶段：1）表示融合与学习，2）重建训练。首先，利用预训练模块构建能够融合辅助文本信息的网络，学习基于语音-文本的联合表示。随后，采用自编码器风格的重建策略，在不增加模型复杂度的情况下，使电喉语音到正常语音转换模型继承这些表示。我们引入三种融合策略，包括中间层融合、输入层融合和混合层融合，逐步增强学习效果。此外，除标准的序列到序列语音转换目标外，还引入对联合表示的额外重建损失，以优化表示迁移。结果：在不同电喉语音到正常语音转换数据集上的实验一致表明，结合数据增强后，我们的方法优于仅依赖语音表示的基线方法。此外，系统设计深度的逐步改进验证了方法的有效性。意义：所提方法为电喉语音增强及辅助通信技术提供了一种可扩展且实用的方法论。