We study automated intrusion detection in an IT infrastructure, specifically the problem of identifying the start of an attack, the type of attack, and the sequence of actions an attacker takes, based on continuous measurements from the infrastructure. We apply statistical learning methods, including Hidden Markov Model (HMM), Long Short-Term Memory (LSTM), and Random Forest Classifier (RFC) to map sequences of observations to sequences of predicted attack actions. In contrast to most related research, we have abundant data to train the models and evaluate their predictive power. The data comes from traces we generate on an in-house testbed where we run attacks against an emulated IT infrastructure. Central to our work is a machine-learning pipeline that maps measurements from a high-dimensional observation space to a space of low dimensionality or to a small set of observation symbols. Investigating intrusions in offline as well as online scenarios, we find that both HMM and LSTM can be effective in predicting attack start time, attack type, and attack actions. If sufficient training data is available, LSTM achieves higher prediction accuracy than HMM. HMM, on the other hand, requires less computational resources and less training data for effective prediction. Also, we find that the methods we study benefit from data produced by traditional intrusion detection systems like SNORT.

翻译：我们研究IT基础设施中的自动化入侵检测问题，具体包括基于基础设施的连续测量数据，识别攻击起始时间、攻击类型以及攻击者采取的攻击动作序列。我们应用包括隐马尔可夫模型（HMM）、长短期记忆网络（LSTM）和随机森林分类器（RFC）在内的统计学习方法，将观测序列映射为预测的攻击动作序列。与大多数相关研究不同，我们拥有充足数据来训练模型并评估其预测能力。这些数据源自内部测试平台生成的痕迹数据，我们在模拟的IT基础设施上执行攻击并采集相关数据。本工作的核心是一个机器学习流水线，能够将高维观测空间的测量数据映射到低维空间或少量观测符号。通过对离线与在线场景下的入侵分析，我们发现HMM和LSTM均能有效预测攻击起始时间、攻击类型及攻击动作。在训练数据充足时，LSTM的预测准确率高于HMM；而HMM则需更少的计算资源和训练数据即可实现有效预测。此外，研究显示我们所采用的方法能受益于传统入侵检测系统（如SNORT）生成的数据。