We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same convergence guarantees when applied to SGD training. We apply approximate tensor operations to single and multi-node training of MLP and CNN networks on MNIST, CIFAR-10 and ImageNet datasets. We demonstrate up to 66% reduction in the amount of computations and communication, and up to 1.37x faster training time while maintaining negligible or no impact on the final test accuracy.

翻译：我们提出了一种更快速深层神经网络培训的新技术,该技术系统地对组成抗拉作业,即矩阵倍增和变异采用基于样本的近似法。我们采用了新的取样技术,研究其理论特性,并证明在应用SGD培训时,它们提供了同样的趋同保障。我们将近似高温操作用于MNIST、CIFAR-10和图像网络数据集对MLP和CNN网络进行单节点和多点培训。我们显示计算和通信量减少了66%,培训时间加快了1.37倍,同时对最终测试准确性没有产生微小或任何影响。