Context: The combination of distributed stream processing with microservice architectures is an emerging pattern for building data-intensive software systems. In such systems, stream processing frameworks such as Apache Flink, Apache Kafka Streams, Apache Samza, Hazelcast Jet, or the Apache Beam SDK are used inside microservices to continuously process massive amounts of data in a distributed fashion. While all of these frameworks promote scalability as a core feature, there is only little empirical research evaluating and comparing their scalability. Objective: The goal of this study to obtain evidence about the scalability of state-of-the-art stream processing framework in different execution environments and regarding different scalability dimensions. Method: We benchmark five modern stream processing frameworks regarding their scalability using a systematic method. We conduct over 740 hours of experiments on Kubernetes clusters in the Google cloud and in a private cloud, where we deploy up to 110 simultaneously running microservice instances, which process up to one million messages per second. Results: All benchmarked frameworks exhibit approximately linear scalability as long as sufficient cloud resources are provisioned. However, the frameworks show considerable differences in the rate at which resources have to be added to cope with increasing load. There is no clear superior framework, but the ranking of the frameworks depends on the use case. Using Apache Beam as an abstraction layer still comes at the cost of significantly higher resource requirements regardless of the use case. We observe our results regardless of scaling load on a microservice, scaling the computational work performed inside the microservice, and the selected cloud environment. Moreover, vertical scaling can be a complementary measure to achieve scalability of stream processing frameworks.

翻译：背景：分布式流处理与微服务架构的结合正成为构建数据密集型软件系统的新兴模式。在此类系统中，Apache Flink、Apache Kafka Streams、Apache Samza、Hazelcast Jet 或 Apache Beam SDK 等流处理框架被部署于微服务内部，以分布式方式持续处理海量数据。尽管这些框架均将可扩展性作为核心特性宣称，但评估和比较其可扩展性的实证研究仍十分匮乏。目标：本研究旨在获取不同执行环境下、面向不同可扩展性维度的最新流处理框架的可扩展性证据。方法：采用系统性方法对五个现代流处理框架的可扩展性进行基准测试。我们在谷歌云和私有云的 Kubernetes 集群上进行了超过740小时的实验，部署了多达110个同时运行的微服务实例，每秒处理消息量最高可达一百万条。结果：只要配置了充足的云资源，所有被测框架均呈现近似线性可扩展性。然而，不同框架在应对负载增加所需资源增速方面存在显著差异。没有绝对占优的框架，框架排名取决于具体用例。无论何种用例，使用 Apache Beam 作为抽象层仍会显著增加资源需求。该结论在微服务负载缩放、微服务内部计算工作量缩放及不同云环境下均保持一致。此外，垂直扩展可作为实现流处理框架可扩展性的补充手段。