Task-based programming models have proven to be a robust and versatile way to approach development of applications for distributed environments. They provide natural programming patterns with high performance. However, execution on this paradigm can be very sensitive to granularity --i.e., the quantity and execution length of tasks. Granularity is often linked with the block size of the data, and finding the optimal block size has several challenges, as it requires inner knowledge of the computing environment. Our proposal is to supplement the task-based programming model with a new mechanism --our SplIter proposal. At its core, the SplIter provides a transparent way to split a collection into partitions (logical groups of blocks, obtained without any transfers nor data rearrangement), which can then be iterated. Tasks are linked to those partitions, which means that SplIter breaks the dependency between block size and task granularity. The evaluation shows that the SplIter is able to achieve performance improvements of over one order of magnitude when compared to the baseline, and it is either competitive or strictly better (depending on application characteristics) to the competitor alternative. We have chosen different applications covering a wide variety of scenarios; those applications are representatives of a broader set of applications and domains. The changes required in the source code of a task-based application are minimal, preserving the high programmability of the programming model. Two different state-of-the-art task-based frameworks have been evaluated for all the applications: COMPSs and Dask, showing that the SplIter can be effectively used within different frameworks.

翻译：基于任务的编程模型已被证明是一种鲁棒且通用的方法，适用于分布式环境下的应用开发。它提供了自然的编程模式并兼具高性能。然而，基于该范式的执行对粒度（即任务的数量和执行长度）非常敏感。粒度通常与数据块大小相关，而寻找最优块大小面临多项挑战，因为这需要了解计算环境的内在知识。我们的方案是为基于任务的编程模型补充一种新机制——SplIter方案。其核心在于，SplIter提供了一种透明的方式将集合分割为分区（无需任何传输或数据重排即可获得的逻辑块组），随后可对这些分区进行迭代。任务与这些分区相关联，这意味着SplIter打破了块大小与任务粒度之间的依赖关系。评估表明，与基线相比，SplIter能够实现超过一个数量级的性能提升，并且与替代方案相比，根据应用特性不同，其性能具有竞争力或严格更优。我们选取了覆盖多种场景的不同应用，这些应用代表了更广泛的应用程序和领域。在基于任务的应用程序中，所需对源代码的修改极小，从而保持了编程模型的高可编程性。针对所有应用，我们对两种最先进的基于任务的框架（COMPSs和Dask）进行了评估，结果表明SplIter可在不同框架中有效使用。