Neural-networks-driven intelligent data-plane (NN-driven IDP) is becoming an emerging topic for excellent accuracy and high performance. Meanwhile we argue that NN-driven IDP should satisfy three design goals: the flexibility to support various NNs models, the low-latency-high-throughput inference performance, and the data-plane-unawareness harming no performance and functionality. Unfortunately, existing work either over-modify NNs for IDP, or insert inline pipelined accelerators into the data-plane, failing to meet the flexibility and unawareness goals. In this paper, we propose Kaleidoscope, a flexible and high-performance co-processor located at the bypass of the data-plane. To address the challenge of meeting three design goals, three key techniques are presented. The programmable run-to-completion accelerators are developed for flexible inference. To further improve performance, we design a scalable inference engine which completes low-latency and low-cost inference for the mouse flows, and perform complex NNs with high-accuracy for the elephant flows. Finally, raw-bytes-based NNs are introduced, which help to achieve unawareness. We prototype Kaleidoscope on both FPGA and ASIC library. In evaluation on six NNs models, Kaleidoscope reaches 256-352 ns inference latency and 100 Gbps throughput with negligible influence on the data-plane. The on-board tested NNs perform state-of-the-art accuracy among other NN-driven IDP, exhibiting the the significant impact of flexibility on enhancing traffic analysis accuracy.

翻译：神经网络驱动的智能数据平面因其卓越的准确性和高性能正成为一个新兴的研究方向。同时，我们认为神经网络驱动的智能数据平面应满足三个设计目标：支持多种神经网络模型的灵活性、低延迟高吞吐量的推理性能，以及对数据平面无感知（即不影响其性能与功能）。遗憾的是，现有工作要么过度修改神经网络以适应智能数据平面，要么将内联流水线加速器插入数据平面，均未能同时满足灵活性与无感知目标。本文提出Kaleidoscope，一种位于数据平面旁路的灵活、高性能协处理器。为应对同时满足三个设计目标的挑战，我们提出了三项关键技术。首先，开发了可编程的“运行至完成”加速器以实现灵活的推理。为进一步提升性能，我们设计了一种可扩展的推理引擎，该引擎对小鼠流完成低延迟、低成本的推理，并对大象流执行高精度的复杂神经网络。最后，引入了基于原始字节的神经网络，这有助于实现无感知特性。我们在FPGA和ASIC库上对Kaleidoscope进行了原型实现。在六种神经网络模型上的评估表明，Kaleidoscope实现了256-352纳秒的推理延迟和100 Gbps的吞吐量，且对数据平面的影响可忽略不计。板载测试的神经网络在其他神经网络驱动的智能数据平面方案中达到了最先进的准确率，这彰显了灵活性对提升流量分析准确性的显著影响。