Robustness to bit errors is a key requirement for the reliable use of neural networks (NNs) on emerging approximate computing platforms and error-prone memory technologies. A common approach to achieve bit error tolerance in NNs is injecting bit flips during training according to a predefined error model. While effective in certain scenarios, training-time bit flip injection introduces substantial computational overhead, often degrades inference accuracy at high error rates, and scales poorly for larger NN architectures. These limitations make error injection an increasingly impractical solution for ensuring robustness on future approximate computing platforms and error-prone memory technologies. In this work, we investigate the mechanisms that enable NNs to tolerate bit errors without relying on error-aware training. We establish a direct connection between bit error tolerance and classification margins at the output layer. Building on this insight, we propose a novel loss function, the Margin Cross-Entropy Loss (MCEL), which explicitly promotes logit-level margin separation while preserving the favorable optimization properties of the standard cross-entropy loss. Furthermore, MCEL introduces an interpretable margin parameter that allows robustness to be tuned in a principled manner. Extensive experimental evaluations across multiple datasets of varying complexity, diverse NN architectures, and a range of quantization schemes demonstrate that MCEL substantially improves bit error tolerance, up to 15 % in accuracy for an error rate of 1 %. Our proposed MCEL method is simple to implement, efficient, and can be integrated as a drop-in replacement for standard CEL. It provides a scalable and principled alternative to training-time bit flip injection, offering new insights into the origins of NN robustness and enabling more efficient deployment on approximate computing and memory systems.

翻译：对位错误的鲁棒性是神经网络（NNs）在新型近似计算平台和易错存储技术上可靠应用的关键要求。在神经网络中实现位错误容错的一种常见方法是根据预定义的错误模型在训练期间注入位翻转。尽管在某些场景下有效，但训练时位翻转注入会引入大量计算开销，在高错误率下通常会降低推理精度，并且对于较大的神经网络架构扩展性较差。这些限制使得错误注入作为确保未来近似计算平台和易错存储技术鲁棒性的方案日益不切实际。在本工作中，我们研究了使神经网络能够容忍位错误而无需依赖错误感知训练的机制。我们在输出层的分类间隔与位错误容错性之间建立了直接联系。基于这一洞见，我们提出了一种新颖的损失函数——间隔交叉熵损失（MCEL），它在保持标准交叉熵损失良好优化特性的同时，显式地促进逻辑值层面的间隔分离。此外，MCEL引入了一个可解释的间隔参数，允许以原则化的方式调整鲁棒性。在多个不同复杂度数据集、多样化神经网络架构以及一系列量化方案上的广泛实验评估表明，MCEL显著提升了位错误容错性，在1%错误率下准确率最高可提升15%。我们提出的MCEL方法实现简单、高效，可作为标准CEL的直接替代方案集成使用。它为训练时位翻转注入提供了一种可扩展且原则化的替代方案，为神经网络鲁棒性的起源提供了新的见解，并支持在近似计算和存储系统上更高效的部署。