Promptly discovering unknown network attacks is critical for reducing the risk of major loss imposed on system or equipment. This paper aims to develop an open-set intrusion detection model to classify known attacks as well as inferring unknown ones. To achieve this, we employ OpenMax and variational autoencoder to propose a dual detection model, VAEMax. First, we extract flow payload feature based on one-dimensional convolutional neural network. Then, the OpenMax is used to classify flows, during which some unknown attacks can be detected, while the rest are misclassified into a certain class of known flows. Finally, use VAE to perform secondary detection on each class of flows, and determine whether the flow is an unknown attack based on the reconstruction loss. Experiments performed on dataset CIC-IDS2017 and CSE-CIC-IDS2018 show our approach is better than baseline models and can be effectively applied to realistic network environments.

翻译：及时识别未知网络攻击对降低系统或设备重大损失风险至关重要。本文旨在开发一种开放集入侵检测模型，既能分类已知攻击，又能推断未知攻击。为此，我们结合OpenMax与变分自编码器，提出了一种双检测模型VAEMax。首先，基于一维卷积神经网络提取流载荷特征；其次，利用OpenMax对流进行分类，在此过程中可检测部分未知攻击，而其余未知攻击被误分类为某类已知流；最后，使用变分自编码器对每类流进行二次检测，并根据重构损失判定该流是否为未知攻击。在CIC-IDS2017和CSE-CIC-IDS2018数据集上的实验表明，我们的方法优于基线模型，可有效应用于实际网络环境。