This work investigates the impact of severe class imbalance on the performance of automated machine learning (AutoML) frameworks for multiclass network intrusion detection using the NSL-KDD dataset. Unlike previous studies that simplify the problem through binary classification or minority-class removal, we preserve the original five-class distribution, including highly underrepresented attacks such as R2L and U2R, enabling a realistic evaluation of imbalance-sensitive learning behavior. Nine open-source AutoML frameworks were analyzed under a unified and reproducible experimental protocol, considering differences in architectural design, ensemble strategies, validation procedures, hyperparameter optimization, and imbalance-handling mechanisms. The results demonstrate that frameworks incorporating ensemble learning and imbalance-aware optimization achieve better minority-class discrimination. PyCaret obtained the best overall performance, reaching 66\% macro-F1, followed by AutoGluon with 55\%, whereas frameworks lacking native balancing support exhibited significant degradation in minority-class detection capability. The analysis further shows that accuracy-oriented optimization alone is insufficient for highly imbalanced IDS scenarios, since high-weighted metrics may coexist with poor generalization on rare attack categories. As a contribution, this work establishes a standardized benchmark for AutoML-based intrusion detection under severe multiclass imbalance, highlighting current architectural limitations and the need for native integration of imbalance-aware optimization, resampling, and stratified evaluation strategies into automated learning pipelines. The source code is publicly available.
翻译:本研究探讨了在使用NSL-KDD数据集进行多类网络入侵检测时,严重类别不平衡对自动化机器学习(AutoML)框架性能的影响。与以往通过二分类或剔除少数类别来简化问题的研究不同,我们保留了原始的五个类别分布,包括高度代表性的攻击类型(如R2L和U2R),从而实现了对不平衡敏感学习行为的现实评估。在统一且可重复的实验协议下,分析了九种开源AutoML框架,考虑了架构设计、集成策略、验证流程、超参数优化以及不平衡处理机制等方面的差异。结果表明,集成学习与不平衡感知优化相结合的框架在少数类别判别上表现更优。PyCaret取得了最佳整体性能,宏F1值达到66%,紧随其后的是AutoGluon(55%),而缺乏原生平衡支持的框架在少数类别检测能力上出现显著下降。进一步分析表明,仅以准确率优化为导向无法应对高度不平衡的入侵检测场景,因为高加权指标可能与罕见攻击类别上的泛化能力差并存。作为贡献,本研究为严重多类不平衡条件下的AutoML入侵检测建立了标准化基准,揭示了当前架构的局限性,以及将不平衡感知优化、重采样和分层评估策略原生集成到自动化学习流水线的必要性。源代码已公开提供。