Dataflow computing was shown to bring significant benefits to multiple niches of systems engineering and has the potential to become a general-purpose paradigm of choice for data-driven application development. One of the characteristic features of dataflow computing is the natural access to the dataflow graph of the entire system. Recently it has been observed that these dataflow graphs can be treated as complete graphical causal models, opening opportunities to apply causal inference techniques to dataflow systems. In this demonstration paper we aim to provide the first practical validation of this idea with a particular focus on causal fault localisation. We provide multiple demonstrations of how causal inference can be used to detect software bugs and data shifts in multiple scenarios with three modern dataflow engines.
翻译:数据流计算已被证明能够为系统工程多个细分领域带来显著优势,并有可能成为数据驱动应用开发的通用计算范式。数据流计算的特征之一在于能够自然获取整个系统的数据流图。近期研究发现,这些数据流图可被视为完整的图结构因果模型,为将因果推断技术应用于数据流系统开辟了新的可能性。本演示论文旨在首次通过实践验证这一理念,重点聚焦于因果故障定位。我们通过多个场景案例,展示了如何利用因果推断检测软件缺陷和数据漂移,并基于三种现代数据流引擎进行了实证演示。