With the increasing inference cost of machine learning models, there is a growing interest in models with fast and efficient inference. Recently, an approach for learning logic gate networks directly via a differentiable relaxation was proposed. Logic gate networks are faster than conventional neural network approaches because their inference only requires logic gate operators such as NAND, OR, and XOR, which are the underlying building blocks of current hardware and can be efficiently executed. We build on this idea, extending it by deep logic gate tree convolutions, logical OR pooling, and residual initializations. This allows scaling logic gate networks up by over one order of magnitude and utilizing the paradigm of convolution. On CIFAR-10, we achieve an accuracy of 86.29% using only 61 million logic gates, which improves over the SOTA while being 29x smaller.

翻译：随着机器学习模型推理成本的不断攀升，人们对具有快速高效推理能力的模型日益关注。近期，一种通过可微分松弛直接学习逻辑门网络的方法被提出。逻辑门网络比传统神经网络方法更快，因为其推理仅需使用NAND、OR和XOR等逻辑门运算符，这些运算符是当前硬件的基础构建模块，能够高效执行。我们基于这一思想，通过深度逻辑门树卷积、逻辑OR池化以及残差初始化进行扩展。这使得逻辑门网络的规模可扩展一个数量级以上，并能利用卷积范式。在CIFAR-10数据集上，我们仅使用6100万个逻辑门就实现了86.29%的准确率，在比现有最佳模型缩小29倍的同时提升了性能。