Binary neural networks (BNNs) have received ever-increasing popularity for their great capability of reducing storage burden as well as quickening inference time. However, there is a severe performance drop compared with {real-valued} networks, due to its intrinsic frequent weight oscillation during training. In this paper, we introduce a Resilient Binary Neural Network (ReBNN) to mitigate the frequent oscillation for better BNNs' training. We identify that the weight oscillation mainly stems from the non-parametric scaling factor. To address this issue, we propose to parameterize the scaling factor and introduce a weighted reconstruction loss to build an adaptive training objective. %To the best of our knowledge, it is the first work to solve BNNs based on a dynamically re-weighted loss function. For the first time, we show that the weight oscillation is controlled by the balanced parameter attached to the reconstruction loss, which provides a theoretical foundation to parameterize it in back propagation. Based on this, we learn our ReBNN by {calculating} the {balanced} parameter {based on} its maximum magnitude, which can effectively mitigate the weight oscillation with a resilient training process. Extensive experiments are conducted upon various network models, such as ResNet and Faster-RCNN for computer vision, as well as BERT for natural language processing. The results demonstrate the overwhelming performance of our ReBNN over prior arts. For example, our ReBNN achieves 66.9\% Top-1 accuracy with ResNet-18 backbone on the ImageNet dataset, surpassing existing state-of-the-arts by a significant margin. Our code is open-sourced at https://github.com/SteveTsui/ReBNN.

翻译：二值神经网络（BNNs）因其在减轻存储负担和加速推理时间方面的卓越能力而日益受到关注。然而，与实值网络相比，由于训练过程中权重固有的频繁振荡，其性能存在显著下降。本文提出了一种弹性二值神经网络（ReBNN），以缓解频繁振荡问题，从而改进BNNs的训练。我们发现权重振荡主要源于非参数化缩放因子。为解决此问题，我们提出对缩放因子进行参数化，并引入加权重建损失以构建自适应训练目标。我们首次证明，权重振荡由重建损失中的平衡参数控制，这为其在反向传播中的参数化提供了理论基础。基于此，我们通过根据最大幅度计算平衡参数来学习ReBNN，从而有效缓解权重振荡并实现弹性训练过程。我们在多种网络模型上进行了大量实验，包括用于计算机视觉的ResNet和Faster-RCNN，以及用于自然语言处理的BERT。结果表明，我们的ReBNN在性能上显著优于现有方法。例如，在ImageNet数据集上，采用ResNet-18主干的ReBNN实现了66.9%的Top-1准确率，以显著优势超越现有最优方法。我们的代码已在https://github.com/SteveTsui/ReBNN开源。