Data is a precious resource in today's society, and is generated at an unprecedented and constantly growing pace. The need to store, analyze, and make data promptly available to a multitude of users introduces formidable challenges in modern software platforms. These challenges radically transformed all research fields that gravitate around data management and processing, with the introduction of distributed data-intensive systems that offer new programming models and implementation strategies to handle data characteristics such as its volume, the rate at which it is produced, its heterogeneity, and its distribution. Each data-intensive system brings its specific choices in terms of data model, usage assumptions, synchronization, processing strategy, deployment, guarantees in terms of consistency, fault tolerance, ordering. Yet, the problems data-intensive systems face and the solutions they propose are frequently overlapping. This paper proposes a unifying model that dissects the core functionalities of data-intensive systems, and precisely discusses alternative design and implementation strategies, pointing out their assumptions and implications. The model offers a common ground to understand and compare highly heterogeneous solutions, with the potential of fostering cross-fertilization across research communities and advancing the field. We apply our model by classifying tens of systems: an exercise that brings to interesting observations on the current trends in the domain of data-intensive systems and suggests open research directions.
翻译:数据在当今社会是一种宝贵资源,其生成速度前所未有且持续增长。存储、分析并及时向众多用户提供数据的需求,对现代软件平台提出了巨大挑战。这些挑战彻底改变了围绕数据管理与处理的研究领域,催生了提供新型编程模型和实现策略以应对数据特征(如体量、生成速率、异构性及分布性)的分布式数据密集型系统。每个数据密集型系统在数据模型、使用假设、同步机制、处理策略、部署方式、一致性保证、容错机制及顺序性等方面均有特定选择。然而,这些系统面临的问题及其提出的解决方案往往存在重叠。本文提出一种统一模型,剖析数据密集型系统的核心功能,并精确讨论替代性设计与实现策略,指明其假设及影响。该模型为理解与比较高度异质的解决方案提供共同基础,有望促进研究群体间的交叉融合并推动领域发展。我们通过对数十个系统进行分类来应用该模型:这一实践揭示了数据密集型系统领域的当前趋势,并提出了开放的研究方向。