We present a comprehensive evaluation of the robustness and explainability of ResNet-like models in the context of Unintended Radiated Emission (URE) classification and suggest a new approach leveraging Neural Stochastic Differential Equations (SDEs) to address identified limitations. We provide an empirical demonstration of the fragility of ResNet-like models to Gaussian noise perturbations, where the model performance deteriorates sharply and its F1-score drops to near insignificance at 0.008 with a Gaussian noise of only 0.5 standard deviation. We also highlight a concerning discrepancy where the explanations provided by ResNet-like models do not reflect the inherent periodicity in the input data, a crucial attribute in URE detection from stable devices. In response to these findings, we propose a novel application of Neural SDEs to build models for URE classification that are not only robust to noise but also provide more meaningful and intuitive explanations. Neural SDE models maintain a high F1-score of 0.93 even when exposed to Gaussian noise with a standard deviation of 0.5, demonstrating superior resilience to ResNet models. Neural SDE models successfully recover the time-invariant or periodic horizontal bands from the input data, a feature that was conspicuously missing in the explanations generated by ResNet-like models. This advancement presents a small but significant step in the development of robust and interpretable models for real-world URE applications where data is inherently noisy and assurance arguments demand interpretable machine learning predictions.
翻译:本文对类ResNet模型在无意识辐射发射(URE)分类任务中的鲁棒性和可解释性进行了全面评估,并提出了一种基于神经随机微分方程(Neural SDEs)的新方法以解决现有局限性。我们通过实验证明了类ResNet模型对高斯噪声扰动的脆弱性:当施加标准差仅为0.5的高斯噪声时,模型性能急剧恶化,F1分数下降至接近零值0.008。同时发现一个令人担忧的差异——类ResNet模型生成的解释未能反映输入数据中固有的周期性特征,而这正是稳定设备URE检测的关键属性。针对上述发现,我们提出将神经随机微分方程创新性地应用于URE分类模型构建,使其不仅具备抗噪声鲁棒性,还能提供更有意义且直观的解释。即使在标准差为0.5的高斯噪声干扰下,神经SDE模型仍能保持0.93的高F1分数,展现出相较于ResNet模型更优越的抗干扰能力。该模型成功从输入数据中恢复出时不变或周期性的水平频带特征,而类ResNet模型生成的解释中显著缺失该特征。这项进展为开发面向实际URE应用的鲁棒可解释模型迈出了虽小但重要的一步——在实际应用中数据本质含噪,且保障论证要求机器学习预测具有可解释性。