Fine-grained network telemetry is becoming a modern datacenter standard and is the basis of essential applications such as congestion control, load balancing, and advanced troubleshooting. As network size increases and telemetry gets more fine-grained, there is a tremendous growth in the amount of data needed to be reported from switches to collectors to enable network-wide view. As a consequence, it is progressively hard to scale data collection systems. We introduce Direct Telemetry Access (DTA), a solution optimized for aggregating and moving hundreds of millions of reports per second from switches into queryable data structures in collectors' memory. DTA is lightweight and it is able to greatly reduce overheads at collectors. DTA is built on top of RDMA, and we propose novel and expressive reporting primitives to allow easy integration with existing state-of-the-art telemetry mechanisms such as INT or Marple. We show that DTA significantly improves telemetry collection rates. For example, when used with INT, it can collect and aggregate over 400M reports per second with a single server, improving over the Atomic MultiLog by up to $16$x.

翻译：精细粒度网络遥测正成为现代数据中心的标准，也是拥塞控制、负载均衡和高级故障排除等关键应用的基础。随着网络规模的扩大和遥测粒度的细化，需要从交换机上报至采集器以实现全网视图的数据量急剧增长，因此数据采集系统的扩展性面临严峻挑战。我们提出直接遥测访问（DTA），这是一种针对聚合与迁移每秒数亿条上报记录而优化的解决方案，可将数据直接存入采集器内存中可查询的数据结构。DTA采用轻量级设计，能显著降低采集器的开销。该方案基于RDMA构建，并提出了新颖且富有表现力的上报原语，可与INT、Marple等现有先进遥测机制无缝集成。实验表明，DTA能大幅提升遥测采集速率：例如，与INT结合使用时，单台服务器即可每秒采集并聚合超过4亿条上报记录，性能较原子多日志提升了最高16倍。