Modern machine learning systems represent their computations as dataflow graphs. The increasingly complex neural network architectures crave for more powerful yet efficient programming abstractions. In this paper we propose an efficient technique for supporting recursive function definitions in dataflow-based systems such as TensorFlow. The proposed approach transforms the given recursive definitions into a static dataflow graph that is enriched with two simple yet powerful dataflow operations. Since static graphs do not change during execution, they can be easily partitioned and executed efficiently in distributed and heterogeneous environments. The proposed technique makes heavy use of the idea of tagging, which was one of the cornerstones of dataflow systems since their inception. We demonstrate that our technique is compatible with the idea of automatic differentiation, a notion that is crucial for dataflow systems that focus on deep learning applications. We describe the principles of an actual implementation of the technique in the TensorFlow framework, and present experimental results that demonstrate that the use of tagging is of paramount importance for developing efficient high-level abstractions for modern dataflow systems.

翻译：现代机器学习系统将其计算过程表示为数据流图。日益复杂的神经网络架构迫切需要更强大且高效的编程抽象。本文提出了一种在基于数据流的系统（如TensorFlow）中支持递归函数定义的高效技术。所提出的方法将给定的递归定义转换为静态数据流图，该图通过两种简单而强大的数据流操作进行增强。由于静态图在执行过程中不会改变，它们可以轻松分区并在分布式异构环境中高效执行。该技术大量运用了标记思想，这是数据流系统自诞生以来的基石之一。我们证明了该技术与自动微分思想兼容，这一概念对于专注于深度学习应用的数据流系统至关重要。我们描述了该技术在TensorFlow框架中的实际实现原理，并通过实验结果表明，标记技术的运用对于为现代数据流系统开发高效的高级抽象具有至关重要的意义。