In this position paper we argue for standardizing how we share and process data in scientific workflows at the network-level to maximize step re-use and workflow portability across platforms and networks in pursuit of a foundational workflow stack. We look to evolve workflows from steps connected point-to-point in a directed acyclic graph (DAG) to steps connected via shared channels in a message system implemented as a network service. To start this evolution, we contribute: a preliminary reference model, architecture, and open tools to implement the architecture today. Our goal stands to improve the deployment and operation of complex workflows by decoupling data sharing and data processing in workflow steps. We seek the workflow community's input on this approach's merit, related research to explore and initial requirements from the workflows community to inform future research.
翻译:在本立场文件中,我们主张在网络层面标准化科学工作流中的数据共享与处理方式,以最大化步骤复用性和工作流在不同平台及网络间的可移植性,从而构建基础性工作流栈。我们致力于推动工作流从有向无环图(DAG)中的点对点连接步骤,演进为通过作为网络服务实现的消息系统中共享通道连接的步骤。为启动这一演进,我们贡献了:一个初步参考模型、架构及当前可实施该架构的开放工具。我们的目标是通过解耦工作流步骤中的数据共享与数据处理,改进复杂工作流的部署与运行。我们期待工作流社区就此方法的优点、待探索的相关研究以及来自工作流社区的初始需求提供反馈,以指导未来研究方向。