Serverless computing offers attractive scalability, elasticity and cost-effectiveness. However, constraints on memory, CPU and function runtime have hindered its adoption for data-intensive applications and machine learning (ML) workloads. Traditional 'server-ful' platforms enable distributed computation via fast networks and well-established inter-process communication (IPC) mechanisms such as MPI and shared memory. In the absence of such solutions in the serverless domain, parallel computation with significant IPC requirements is challenging. We present FSD-Inference, the first fully serverless and highly scalable system for distributed ML inference. We explore potential communication channels, in conjunction with Function-as-a-Service (FaaS) compute, to design a state-of-the-art solution for distributed ML within the context of serverless data-intensive computing. We introduce novel fully serverless communication schemes for ML inference workloads, leveraging both cloud-based publish-subscribe/queueing and object storage offerings. We demonstrate how publish-subscribe/queueing services can be adapted for FaaS IPC with comparable performance to object storage, while offering significantly reduced cost at high parallelism levels. We conduct in-depth experiments on benchmark DNNs of various sizes. The results show that when compared to server-based alternatives, FSD-Inference is significantly more cost-effective and scalable, and can even achieve competitive performance against optimized HPC solutions. Experiments also confirm that our serverless solution can handle large distributed workloads and leverage high degrees of FaaS parallelism.

翻译：无服务器计算具有出色的可扩展性、弹性和成本效益。然而，内存、CPU和函数运行时的限制阻碍了其在数据密集型应用和机器学习工作负载中的采用。传统“有服务器”平台通过快速网络和成熟的进程间通信机制（如MPI和共享内存）实现分布式计算。在无服务器领域缺乏此类解决方案的情况下，具有显著IPC需求的并行计算面临挑战。我们提出了FSD-Inference，这是首个完全无服务器且高度可扩展的分布式机器学习推理系统。我们探索了潜在的通信通道，并结合函数即服务计算，设计了一种在无服务器数据密集型计算背景下进行分布式机器学习的先进解决方案。针对机器学习推理工作负载，我们引入了新颖的完全无服务器通信方案，利用了基于云的发布-订阅/队列和对象存储服务。我们展示了发布-订阅/队列服务如何适应FaaS的进程间通信，其性能与对象存储相当，同时在高并行度下显著降低成本。我们在不同规模的基准深度神经网络上进行了深入实验。结果表明，与基于服务器的替代方案相比，FSD-Inference在成本效益和可扩展性方面显著更优，甚至能够与优化的HPC解决方案竞争性能。实验还证实，我们的无服务器解决方案可以处理大规模分布式工作负载，并充分利用高程度的FaaS并行性。