Due to the popularity of the FaaS programming model, there is now a wide variety of commercial and open-source FaaS systems. Hence, for comparison of different FaaS systems and their configuration options, FaaS application developers rely on FaaS benchmarking frameworks. Existing frameworks, however, tend to evaluate only single isolated aspects, a more holistic application-centric benchmarking framework is still missing. In previous work, we proposed BeFaaS, an extensible application-centric benchmarking framework for FaaS environments that focuses on the evaluation of FaaS platforms through realistic and typical examples of FaaS applications. In this extended paper, we (i) enhance our benchmarking framework with additional features for distributed FaaS setups, (ii) design application benchmarks reflecting typical FaaS use cases, and (iii) use them to run extensive experiments with commercial cloud FaaS platforms (AWS Lambda, Azure Functions, Google Cloud Functions) and the tinyFaaS edge serverless platform. BeFaaS now includes four FaaS application-centric benchmarks, is extensible for additional workload profiles and platforms, and supports federated benchmark runs in which the benchmark application is distributed over multiple FaaS systems while collecting fine-grained measurement results for drill-down analysis. Our experiment results show that (i) network transmission is a major contributor to response latency for function chains, (ii) this effect is exacerbated in hybrid edge-cloud deployments, (iii) the trigger delay between a published event and the start of the triggered function ranges from about 100ms for AWS Lambda to 800ms for Google Cloud Functions, and (iv) Azure Functions shows the best cold start behavior for our workloads.

翻译：由于FaaS编程模型的普及，目前市面上涌现出多种商业与开源FaaS系统。因此，为比较不同FaaS系统及其配置选项，FaaS应用开发者需依赖基准评测框架。然而现有框架往往仅评估单一孤立维度，仍缺乏更全面的面向应用的基准评测框架。在前序工作中，我们提出了BeFaaS——一个面向FaaS环境的可扩展应用级基准评测框架，通过真实且典型的FaaS应用示例评估平台性能。本文作为扩展研究：(i) 为分布式FaaS场景新增特性以增强基准框架，(ii) 设计反映典型FaaS用例的应用基准测试，(iii) 利用这些测试对商业云FaaS平台（AWS Lambda、Azure Functions、Google Cloud Functions）及edge serverless平台tinyFaaS开展大规模实验。BeFaaS现包含四项FaaS应用级基准测试，支持扩展额外工作负载配置与平台类型，并支持联邦式基准运行——即基准应用分布式部署于多个FaaS系统，同时采集细粒度测量结果以进行下钻分析。实验结果表明：(i) 网络传输是函数链响应延迟的主要贡献因素；(ii) 该效应在混合边缘-云部署场景中进一步加剧；(iii) 事件发布与触发函数启动之间的延迟，从AWS Lambda的约100ms到Google Cloud Functions的800ms不等；(iv) 在本实验工作负载下，Azure Functions展现出最优的冷启动表现。