It is still a challenging task to learn a neural text generation model under the framework of generative adversarial networks (GANs) since the entire training process is not differentiable. The existing training strategies either suffer from unreliable gradient estimations or imprecise sentence representations. Inspired by the principle of sparse coding, we propose a SparseGAN that generates semantic-interpretable, but sparse sentence representations as inputs to the discriminator. The key idea is that we treat an embedding matrix as an over-complete dictionary, and use a linear combination of very few selected word embeddings to approximate the output feature representation of the generator at each time step. With such semantic-rich representations, we not only reduce unnecessary noises for efficient adversarial training, but also make the entire training process fully differentiable. Experiments on multiple text generation datasets yield performance improvements, especially in sequence-level metrics, such as BLEU.

翻译：在生成对抗网络框架下学习神经文本生成模型仍是一项具有挑战性的任务，因为整个训练过程不可微。现有训练策略要么存在不可靠的梯度估计问题，要么存在句子表示不精确的问题。受稀疏编码原理启发，我们提出SparseGAN，该模型生成语义可解释的稀疏句子表示作为判别器的输入。核心思想是将嵌入矩阵视为过完备字典，通过极少数选定词嵌入的线性组合来近似生成器在每个时间步的输出特征表示。借助这种语义丰富的表示，我们不仅能够减少不必要的噪声以实现高效对抗训练，还能使整个训练过程完全可微。在多个文本生成数据集上的实验取得了性能提升，尤其在BLEU等序列级指标上表现显著。