Binary Neural Networks (BNNs), which constrain both weights and activations to binary values, offer substantial reductions in computational complexity, memory footprint, and energy consumption. These advantages make them particularly well suited for deployment on resource-constrained devices. However, training BNNs via gradient-based optimization remains challenging due to the discrete nature of their variables. The dominant approach, quantization-aware training, circumvents this issue by employing surrogate gradients. Yet, this method requires maintaining latent full-precision parameters and performing the backward pass with floating-point arithmetic, thereby forfeiting the efficiency of binary operations during training. While alternative approaches based on local learning rules exist, they are unsuitable for global credit assignment and for back-propagating errors in multi-layer architectures. This paper introduces Binary Error Propagation (BEP), the first learning algorithm to establish a principled, discrete analog of the backpropagation chain rule. This mechanism enables error signals, represented as binary vectors, to be propagated backward through multiple layers of a neural network. BEP operates entirely on binary variables, with all forward and backward computations performed using only bitwise operations. Crucially, this makes BEP the first solution to enable end-to-end binary training for recurrent neural network architectures. We validate the effectiveness of BEP on both multi-layer perceptrons and recurrent neural networks, demonstrating gains of up to +6.89% and +10.57% in test accuracy, respectively. The proposed algorithm is released as an open-source repository.

翻译：二值神经网络（BNNs）将权重和激活值约束为二进制数值，从而显著降低了计算复杂度、内存占用和能耗。这些优势使其特别适合部署在资源受限的设备上。然而，由于变量的离散性，通过基于梯度的优化方法训练BNNs仍然具有挑战性。目前的主流方法——量化感知训练——通过使用代理梯度来规避这一问题。但该方法需要维护潜在的全精度参数，并使用浮点运算执行反向传播过程，从而丧失了训练期间二进制运算的效率。虽然存在基于局部学习规则的替代方法，但它们不适用于全局信用分配，也不适用于多层架构中的误差反向传播。本文提出了二进制误差传播（BEP），这是首个建立反向传播链式法则的离散化原理性模拟的学习算法。该机制使得以二进制向量表示的误差信号能够在神经网络的多层之间向后传播。BEP完全在二进制变量上运行，所有前向和反向计算仅使用按位运算完成。至关重要的是，这使得BEP成为首个能够实现循环神经网络架构端到端二进制训练的解决方案。我们在多层感知机和循环神经网络上验证了BEP的有效性，分别展示了测试准确率高达+6.89%和+10.57%的提升。所提出的算法已作为开源代码库发布。