In this paper, we propose a new approach to train deep learning models using game theory concepts including Generative Adversarial Networks (GANs) and Adversarial Training (AT) where we deploy a double-oracle framework using best response oracles. GAN is essentially a two-player zero-sum game between the generator and the discriminator. The same concept can be applied to AT with attacker and classifier as players. Training these models is challenging as a pure Nash equilibrium may not exist and even finding the mixed Nash equilibrium is difficult as training algorithms for both GAN and AT have a large-scale strategy space. Extending our preliminary model DO-GAN, we propose the methods to apply the double oracle framework concept to Adversarial Neural Architecture Search (NAS for GAN) and Adversarial Training (NAS for AT) algorithms. We first generalize the players' strategies as the trained models of generator and discriminator from the best response oracles. We then compute the meta-strategies using a linear program. For scalability of the framework where multiple network models of best responses are stored in the memory, we prune the weakly-dominated players' strategies to keep the oracles from becoming intractable. Finally, we conduct experiments on MNIST, CIFAR-10 and TinyImageNet for DONAS-GAN. We also evaluate the robustness under FGSM and PGD attacks on CIFAR-10, SVHN and TinyImageNet for DONAS-AT. We show that all our variants have significant improvements in both subjective qualitative evaluation and quantitative metrics, compared with their respective base architectures.

翻译：本文提出一种利用博弈论概念训练深度学习模型的新方法，该方法涵盖生成对抗网络与对抗训练，其中我们通过最佳响应预言机部署了双预言机框架。生成对抗网络本质上是生成器与判别器之间的二人零和博弈。相同概念可应用于对抗训练，其中攻击者与分类器作为博弈参与者。训练此类模型具有挑战性，因为纯纳什均衡可能不存在，且即使寻找混合纳什均衡也十分困难，这是由于生成对抗网络与对抗训练的训练算法均涉及大规模策略空间。基于我们初步模型DO-GAN的扩展，本文提出将双预言机框架概念应用于对抗神经架构搜索与对抗训练算法的方法。我们首先将参与者的策略泛化为来自最佳响应预言机的生成器与判别器训练模型，随后通过线性规划计算元策略。针对框架中多个最佳响应网络模型存储于内存的可扩展性问题，我们通过剪枝弱占优参与者策略来防止预言机变得难以处理。最后，我们在MNIST、CIFAR-10和TinyImageNet数据集上对DONAS-GAN进行实验验证，并在CIFAR-10、SVHN和TinyImageNet数据集上评估DONAS-AT在FGSM与PGD攻击下的鲁棒性。实验表明，与各自基础架构相比，我们提出的所有变体在主观定性评估与定量指标上均取得显著提升。