Cloud infrastructure supports the efficient operation of data pipelines regarding requirements like cost, speed, and resource utilization. We present an integrated view of optimization opportunities for cloud-based data pipelines by conducting a systematic review of existing literature on optimization approaches to cloud infrastructure performance for data pipelines. Our study contributes a theory of optimization goals like minimizing cost, reducing execution time, and cost-makespan trade-offs, consisting of dimensions such as single vs. multi-cloud, batch vs. stream processing, etc. We highlight gaps in primary research, including the underexploration of multi-tenant environments and lack of industry evaluation, and suggest directions for future research.

翻译：云基础设施支撑着数据管道在成本、速度和资源利用率等方面的高效运行。我们通过对现有关于数据管道云基础设施性能优化方法的文献进行系统综述，提出了一个针对基于云的数据管道的优化机遇整合视角。本研究构建了一个优化目标理论，包括成本最小化、执行时间缩短以及成本与完工时间的权衡，其维度涵盖单云与多云、批处理与流处理等。我们指出了主要研究中存在的空白，包括多租户环境探索不足以及缺乏行业评估，并提出了未来研究的方向。