In this paper, we present Flock, a cloud-native streaming query engine that leverages the on-demand elasticity of Function-as-a-Service (FaaS) platforms to perform real-time data analytics. Traditional server-centric deployments often suffer from resource under- or over-provisioning, leading to resource wastage or performance degradation. Flock addresses these issues by providing more fine-grained elasticity that can dynamically match the per-query basis with continuous scaling, and its billing methods are more fine-grained with millisecond granularity, making it a low-cost solution for stream processing. Our approach, payload invocation, eliminates the need for external storage services and eliminates the requirement for a query coordinator in the data architecture. Our evaluation shows that Flock significantly outperforms state-of-the-art systems in terms of cost, especially on ARM processors, making it a promising solution for real-time data analytics on FaaS platforms.
翻译:本文提出Flock,一种云原生流式查询引擎,利用函数即服务(FaaS)平台的按需弹性能力执行实时数据分析。传统的服务器为中心部署模式常面临资源过度或不足配置的问题,导致资源浪费或性能下降。Flock通过提供更细粒度的弹性能力(可动态匹配每查询粒度并实现持续扩展)解决了上述问题,其计费方式采用毫秒级更细粒度模式,成为低成本流处理方案。我们提出的负载调用方法无需外部存储服务,并消除了数据架构中对查询协调器的需求。评估表明,Flock在成本方面显著优于现有最优系统,尤其在ARM处理器上表现突出,使其成为FaaS平台实时数据分析领域的理想解决方案。