Modern vehicles rely on electronic control units (ECUs) interconnected through the Controller Area Network (CAN), making in-vehicle communication a critical security concern. Machine learning (ML)-based intrusion detection systems (IDS) are increasingly deployed to protect CAN traffic, yet their robustness against adversarial manipulation remains largely unexplored. We present a systematic adversarial evaluation of CAN IDS using the ROAD dataset, comparing four shallow learning models with a deep neural network-based detector. Using protocol-compliant, payload-level perturbations generated via FGSM, BIM and PGD, we evaluate adversarial effects on both benign and malicious CAN frames. While all models achieve strong baseline performance under benign conditions, adversarial perturbations reveal substantial vulnerabilities. Although shallow and deep models are robust to false-alarm induction, with the deep neural network (DNN) performing best on benign traffic, all architectures suffer significant increases in missed attacks. Notably, under gradient-based attacks, the shallow model extra trees (ET) demonstrates improved robustness to missed-attack induction compared to the other models. Our results demonstrate that adversarial manipulation can simultaneously trigger false alarms and evade detection, underscoring the need for adversarial robustness evaluation in safety-critical automotive IDS.

翻译：现代车辆依赖通过控制器局域网（CAN）互连的电子控制单元（ECU），使得车内通信成为一个关键的安全问题。基于机器学习（ML）的入侵检测系统（IDS）越来越多地被部署以保护CAN流量，然而其对抗对抗性操纵的鲁棒性在很大程度上仍未得到充分探索。我们使用ROAD数据集对CAN IDS进行了系统的对抗性评估，比较了四种浅层学习模型与一种基于深度神经网络的检测器。通过使用FGSM、BIM和PGD生成的符合协议规范、载荷级别的扰动，我们评估了对抗性扰动对良性及恶意CAN帧的影响。尽管所有模型在良性条件下均表现出强大的基线性能，但对抗性扰动揭示了显著的脆弱性。虽然浅层和深层模型在诱导误报方面具有鲁棒性，且深度神经网络（DNN）在良性流量上表现最佳，但所有架构的漏报攻击均显著增加。值得注意的是，在基于梯度的攻击下，浅层模型极端随机树（ET）在抵抗漏报攻击诱导方面表现出优于其他模型的鲁棒性。我们的结果表明，对抗性操纵可以同时触发误报并逃避检测，这强调了在安全关键型汽车IDS中进行对抗性鲁棒性评估的必要性。