Nowadays, many scientific workflows from different domains, such as Remote Sensing, Astronomy, and Bioinformatics, are executed on large computing infrastructures managed by resource managers. Scientific workflow management systems (SWMS) support the workflow execution and communicate with the infrastructures' resource managers. However, the communication between SWMS and resource managers is complicated by a) inconsistent interfaces between SMWS and resource managers and b) the lack of support for workflow dependencies and workflow-specific properties. To tackle these issues, we developed the Common Workflow Scheduler Interface (CWSI), a simple yet powerful interface to exchange workflow-related information between a SWMS and a resource manager, making the resource manager workflow-aware. The first prototype implementations show that the CWSI can reduce the makespan already with simple but workflow-aware strategies up to 25%. In this paper, we show how existing workflow resource management research can be integrated into the CWSI.
翻译:如今,来自不同领域(如遥感、天文学和生物信息学)的许多科学工作流均在由资源管理器管理的大型计算基础设施上执行。科学工作流管理系统(SWMS)支持工作流的执行,并与基础设施的资源管理器进行通信。然而,SWMS与资源管理器之间的通信因以下问题而变得复杂:a) SWMS与资源管理器之间接口不一致;b) 缺乏对工作流依赖关系及工作流特定属性的支持。为解决这些问题,我们开发了通用工作流调度器接口(CWSI),这是一个简单而强大的接口,用于在SWMS与资源管理器之间交换工作流相关信息,使资源管理器具备工作流感知能力。首个原型实现表明,即便采用简单但具备工作流感知的策略,CWSI也能将完工时间缩短高达25%。本文展示了如何将现有的工作流资源管理研究集成到CWSI中。