Due to the popularity of the FaaS programming model, there is now a wide variety of commercial and open-source FaaS systems. Hence, for comparison of different FaaS systems and their configuration options, FaaS application developers rely on FaaS benchmarking frameworks. Existing frameworks, however, tend to evaluate only single isolated aspects, a more holistic application-centric benchmarking framework is still missing. In previous work, we proposed BeFaaS, an extensible application-centric benchmarking framework for FaaS environments that focuses on the evaluation of FaaS platforms through realistic and typical examples of FaaS applications. In this extended paper, we (i) enhance our benchmarking framework with additional features for distributed FaaS setups, (ii) design application benchmarks reflecting typical FaaS use cases, and (iii) use them to run extensive experiments with commercial cloud FaaS platforms (AWS Lambda, Azure Functions, Google Cloud Functions) and the tinyFaaS edge serverless platform. BeFaaS now includes four FaaS application-centric benchmarks, is extensible for additional workload profiles and platforms, and supports federated benchmark runs in which the benchmark application is distributed over multiple FaaS systems while collecting fine-grained measurement results for drill-down analysis. Our experiment results show that (i) network transmission is a major contributor to response latency for function chains, (ii) this effect is exacerbated in hybrid edge-cloud deployments, (iii) the trigger delay between a published event and the start of the triggered function ranges from about 100ms for AWS Lambda to 800ms for Google Cloud Functions, and (iv) Azure Functions shows the best cold start behavior for our workloads.

翻译：由于FaaS编程模型的普及，如今已涌现出大量商业和开源FaaS系统。为比较不同FaaS系统及其配置选项，FaaS应用开发者需要依赖基准测试框架。然而现有框架往往仅评估单一方面，仍缺少更全面的面向应用的基准测试方法。在前期工作中，我们提出了BeFaaS——一个可扩展的面向应用的FaaS环境基准测试框架，通过真实典型的FaaS应用实例来评估平台性能。在本扩展论文中，我们：(i) 为分布式FaaS场景增强基准测试框架功能，(ii) 设计反映典型FaaS用例的应用基准测试，(iii) 利用商业云FaaS平台（AWS Lambda、Azure Functions、Google Cloud Functions）及tinyFaaS边缘无服务器平台开展大规模实验。BeFaaS现包含四项面向FaaS应用的基准测试，支持扩展更多负载配置与平台，并支持联邦式基准运行——将基准应用分布在多个FaaS系统上，同时采集细粒度测量结果用于深度分析。实验结果表明：(i) 网络传输是函数链响应延迟的主要影响因素，(ii) 混合边缘-云部署会加剧该效应，(iii) 从事件发布到触发函数启动的延迟，AWS Lambda约100ms，Google Cloud Functions约800ms，(iv) 针对本工作负载，Azure Functions的冷启动表现最优。