Noise injection and data augmentation strategies have been effective for enhancing the generalisation and robustness of neural networks (NNs). Certain types of noise such as label smoothing and MixUp have also been shown to improve calibration. Since noise can be added in various stages of the NN's training, it motivates the question of when and where the noise is the most effective. We study a variety of noise types to determine how much they improve calibration and generalisation, and under what conditions. More specifically we evaluate various noise-injection strategies in both in-distribution (ID) and out-of-distribution (OOD) scenarios. The findings highlight that activation noise was the most transferable and effective in improving generalisation, while input augmentation noise was prominent in improving calibration on OOD but not necessarily ID data.

翻译：噪声注入与数据增强策略已有效提升神经网络的泛化能力与鲁棒性。标签平滑和MixUp等特定噪声类型也被证明能改善模型校准。由于噪声可在神经网络训练的不同阶段引入，这引发了一个关键问题：在何时何处添加噪声最为有效？本研究系统考察了多种噪声类型，以明确其在不同条件下对校准和泛化性能的改善程度。具体而言，我们在分布内与分布外场景下评估了多种噪声注入策略。研究结果表明，激活噪声在提升泛化性能方面具有最优的迁移性与有效性，而输入增强噪声在改善分布外数据校准方面表现突出，但对分布内数据的校准提升效果未必显著。