Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere IoT datasets both capture IoT-related attacks, but the deep neural network performed the best when tested using the BoT_IoT dataset while HELAD performed the best when tested using the Stratosphere IoT dataset. So although we found that a deep neural network solution had the highest average F1 scores on tested datasets, it is not always the best-performing one. We further discuss difficulties in using IDS from literature and project repositories, which complicated drawing definitive conclusions regarding IDS selection.

翻译：本文对近期多种入侵检测系统（IDS）进行了实证性比较，旨在通过客观对比帮助用户根据自身需求选择最合适的解决方案。研究结果表明，不存在绝对最优的单一方案，其效果取决于攻击类型、复杂性及数据集中网络环境等外部变量。例如，BoT_IoT与Stratosphere IoT数据集虽均采集物联网相关攻击数据，但深度神经网络在使用BoT_IoT数据集测试时表现最佳，而HELAD在Stratosphere IoT数据集测试中效果最优。因此，尽管深度神经网络方案在所测数据集上获得了最高的平均F1值，它却并非始终表现最优。我们进一步探讨了复现文献及项目仓库中IDS技术存在的困难，这些因素导致难以就IDS选型得出明确结论。