We study the capabilities of GANs and Wasserstein GANs equipped with Transformer encoders to generate sensible and challenging training data for symbolic reasoning domains. We conduct experiments on two problem domains where Transformers have been successfully applied recently: symbolic mathematics and temporal specifications in verification. Even without autoregression, our GAN models produce syntactically correct instances. We show that the generated data can be used as a substitute for real training data when training a classifier, and, especially, that training data can be generated from a dataset that is too small to be trained on directly. Using a GAN setting also allows us to alter the target distribution: We show that by adding a classifier uncertainty part to the generator objective, we obtain a dataset that is even harder to solve for a temporal logic classifier than our original dataset.

翻译：我们研究了配备Transformer编码器的生成对抗网络（GAN）和Wasserstein GAN在符号推理领域生成合理且具有挑战性的训练数据的能力。我们在两个Transformer近期成功应用的领域进行实验：符号数学和验证中的时序规范。即使不使用自回归机制，我们的GAN模型也能生成语法正确的实例。我们证明，生成的训练数据在训练分类器时可作为真实训练数据的替代品，尤其能够从规模过小、无法直接训练的数据集中生成训练数据。采用GAN框架还使我们能够调整目标分布：通过在生成器目标函数中加入分类器不确定性项，我们获得了比原始数据集更难被时序逻辑分类器求解的数据集。