With the increasing prevalence of encrypted network traffic, cyber security analysts have been turning to machine learning (ML) techniques to elucidate the traffic on their networks. However, ML models can become stale as new traffic emerges that is outside of the distribution of the training set. In order to reliably adapt in this dynamic environment, ML models must additionally provide contextualized uncertainty quantification to their predictions, which has received little attention in the cyber security domain. Uncertainty quantification is necessary both to signal when the model is uncertain about which class to choose in its label assignment and when the traffic is not likely to belong to any pre-trained classes. We present a new, public dataset of network traffic that includes labeled, Virtual Private Network (VPN)-encrypted network traffic generated by 10 applications and corresponding to 5 application categories. We also present an ML framework that is designed to rapidly train with modest data requirements and provide both calibrated, predictive probabilities as well as an interpretable "out-of-distribution" (OOD) score to flag novel traffic samples. We describe calibrating OOD scores using p-values of the relative Mahalanobis distance. We demonstrate that our framework achieves an F1 score of 0.98 on our dataset and that it can extend to an enterprise network by testing the model: (1) on data from similar applications, (2) on dissimilar application traffic from an existing category, and (3) on application traffic from a new category. The model correctly flags uncertain traffic and, upon retraining, accurately incorporates the new data.

翻译：随着加密网络流量的日益普及，网络安全分析师开始借助机器学习（ML）技术来解析其网络中的流量。然而，当新出现的流量超出训练集的数据分布时，ML模型可能变得过时。为了在这种动态环境中可靠地适应，ML模型必须额外为其预测提供基于上下文的不确定性量化，而这一点在网络安全领域尚未得到充分关注。不确定性量化对于以下两方面至关重要：一是当模型不确定应选择哪个类别进行标签分配时提供信号，二是当流量可能不属于任何预训练类别时发出警示。我们提出了一个新的公开网络流量数据集，其中包含由10个应用生成的带有标签的、经虚拟专用网络（VPN）加密的网络流量，这些流量对应5个应用类别。我们还提出了一个ML框架，该框架设计为在适度的数据需求下快速训练，并提供校准后的预测概率以及可解释的“分布外”（OOD）得分，以标记新颖的流量样本。我们描述了利用相对马氏距离的p值校准OOD得分的方法。我们证明，该框架在我们的数据集上实现了0.98的F1得分，并且可以通过以下方式扩展至企业网络测试： (1) 针对相似应用的数据， (2) 来自现有类别的不相似应用流量，以及 (3) 来自新类别的应用流量。模型能正确标记不确定的流量，并在重新训练后准确整合新数据。