On-device learning remains a formidable challenge, especially when dealing with resource-constrained devices that have limited computational capabilities. This challenge is primarily rooted in two key issues: first, the memory available on embedded devices is typically insufficient to accommodate the memory-intensive back-propagation algorithm, which often relies on floating-point precision. Second, the development of learning algorithms on models with extreme quantization levels, such as Binary Neural Networks (BNNs), is critical due to the drastic reduction in bit representation. In this study, we propose a solution that combines recent advancements in the field of Continual Learning (CL) and Binary Neural Networks to enable on-device training while maintaining competitive performance. Specifically, our approach leverages binary latent replay (LR) activations and a novel quantization scheme that significantly reduces the number of bits required for gradient computation. The experimental validation demonstrates a significant accuracy improvement in combination with a noticeable reduction in memory requirement, confirming the suitability of our approach in expanding the practical applications of deep learning in real-world scenarios.

翻译：设备端学习仍是一项艰巨挑战，尤其在处理计算能力有限的资源受限设备时。该挑战主要源于两个关键问题：首先，嵌入式设备可用内存通常不足以支持依赖浮点精度的内存密集型的反向传播算法；其次，在极端量化水平的模型（如二值神经网络）上开发学习算法至关重要，因为其比特表示大幅减少。本研究提出一种结合持续学习与二值神经网络领域最新进展的解决方案，在保持竞争性性能的同时实现设备端训练。具体而言，我们的方法利用二值潜重放激活函数和新型量化方案，大幅减少梯度计算所需的比特数。实验验证表明，该方法在显著提升精度的同时显著降低内存需求，证实了该方法在扩展深度学习实际应用场景中的可行性。