$\partial\mathbb{B}$ nets are differentiable neural networks that learn discrete boolean-valued functions by gradient descent. $\partial\mathbb{B}$ nets have two semantically equivalent aspects: a differentiable soft-net, with real weights, and a non-differentiable hard-net, with boolean weights. We train the soft-net by backpropagation and then `harden' the learned weights to yield boolean weights that bind with the hard-net. The result is a learned discrete function. `Hardening' involves no loss of accuracy, unlike existing approaches to neural network binarization. Preliminary experiments demonstrate that $\partial\mathbb{B}$ nets achieve comparable performance on standard machine learning problems yet are compact (due to 1-bit weights) and interpretable (due to the logical nature of the learnt functions).
翻译:$\partial\mathbb{B}$ 网络是可微分的神经网络,能够通过梯度下降学习离散的布尔值函数。$\partial\mathbb{B}$ 网络具有两个语义等价的方面:一个具有实值权重的可微分软网络,以及一个具有布尔权重的不可微分硬网络。我们通过反向传播训练软网络,然后将学习到的权重“硬化”以生成与硬网络结合的布尔权重,最终得到一个学习到的离散函数。与现有的神经网络二值化方法不同,“硬化”过程不会造成精度损失。初步实验表明,$\partial\mathbb{B}$ 网络在标准机器学习问题上取得了可比的性能,同时由于1比特权重的特性而更加紧凑,并且由于所学函数的逻辑特性而具有可解释性。