Data is a precious resource in today's society, and is generated at an unprecedented and constantly growing pace. The need to store, analyze, and make data promptly available to a multitude of users introduces formidable challenges in modern software platforms. These challenges radically transformed all research fields that gravitate around data management and processing, with the introduction of distributed data-intensive systems that offer new programming models and implementation strategies to handle data characteristics such as its volume, the rate at which it is produced, its heterogeneity, and its distribution. Each data-intensive system brings its specific choices in terms of data model, usage assumptions, synchronization, processing strategy, deployment, guarantees in terms of consistency, fault tolerance, ordering. Yet, the problems data-intensive systems face and the solutions they propose are frequently overlapping. This paper proposes a unifying model that dissects the core functionalities of data-intensive systems, and precisely discusses alternative design and implementation strategies, pointing out their assumptions and implications. The model offers a common ground to understand and compare highly heterogeneous solutions, with the potential of fostering cross-fertilization across research communities and advancing the field. We apply our model by classifying tens of systems: an exercise that brings to interesting observations on the current trends in the domain of data-intensive systems and suggests open research directions.
翻译:数据是当今社会的宝贵资源,其产生速度前所未有且持续增长。在现代软件平台中,对数据进行存储、分析并使其能够被众多用户及时访问的需求带来了严峻挑战。这些挑战从根本上改变了围绕数据管理与处理的所有研究领域,催生了分布式数据密集型系统。这类系统提供了新的编程模型和实现策略,以处理数据的规模、产生速率、异构性及分布性等特征。每个数据密集型系统在数据模型、使用假设、同步机制、处理策略、部署方式以及一致性、容错性、排序性等保证方面都有其特定选择。然而,数据密集型系统面临的问题及其提出的解决方案往往存在重叠。本文提出一个统一模型,剖析数据密集型系统的核心功能,并深入探讨不同的设计与实现策略,指出其假设与影响。该模型为理解与对比高度异构的解决方案提供了共同基础,有望促进研究社区间的交叉融合并推动该领域发展。我们通过分类数十个系统来应用该模型:这一实践不仅揭示了数据密集型系统领域的当前趋势,还指出了开放性的研究方向。