Can we develop a model that can synthesize realistic speech directly from a latent space, without explicit conditioning? Despite several efforts over the last decade, previous adversarial and diffusion-based approaches still struggle to achieve this, even on small-vocabulary datasets. To address this, we propose AudioStyleGAN (ASGAN) -- a generative adversarial network for unconditional speech synthesis tailored to learn a disentangled latent space. Building upon the StyleGAN family of image synthesis models, ASGAN maps sampled noise to a disentangled latent vector which is then mapped to a sequence of audio features so that signal aliasing is suppressed at every layer. To successfully train ASGAN, we introduce a number of new techniques, including a modification to adaptive discriminator augmentation which probabilistically skips discriminator updates. We apply it on the small-vocabulary Google Speech Commands digits dataset, where it achieves state-of-the-art results in unconditional speech synthesis. It is also substantially faster than existing top-performing diffusion models. We confirm that ASGAN's latent space is disentangled: we demonstrate how simple linear operations in the space can be used to perform several tasks unseen during training. Specifically, we perform evaluations in voice conversion, speech enhancement, speaker verification, and keyword classification. Our work indicates that GANs are still highly competitive in the unconditional speech synthesis landscape, and that disentangled latent spaces can be used to aid generalization to unseen tasks. Code, models, samples: https://github.com/RF5/simple-asgan/

翻译：能否开发一个无需显式条件控制、直接从潜在空间合成逼真语音的模型？尽管过去十年间已有诸多尝试，但现有的对抗式与扩散模型方法仍难以实现这一目标，即便在小型词汇数据集上也是如此。为此，我们提出AudioStyleGAN（ASGAN）——一种专为学习解耦潜在空间而设计的无条件语音合成生成对抗网络。ASGAN继承StyleGAN系列图像合成模型的设计理念，将采样噪声映射为解耦后的潜在向量，进而将其转换为音频特征序列，从而在每一层抑制信号混叠。为成功训练ASGAN，我们引入多项新技术，包括对自适应判别器增强的改良——通过概率性跳过判别器更新。在小型词汇数据集Google Speech Commands数字子集上的实验表明，该方法在无条件语音合成任务中达到业界最优水平，且推理速度显著优于现有顶尖扩散模型。我们证实ASGAN的潜在空间具备解耦特性：仅通过该空间中的简单线性运算即可完成多项训练时未见任务。具体而言，我们在语音转换、语音增强、说话人验证及关键词分类四项任务中进行了评估。本研究表明，GAN在无条件语音合成领域仍具有强大竞争力，而解耦潜在空间可有效提升对未见任务的泛化能力。代码、模型与示例音频详见：https://github.com/RF5/simple-asgan/