Schema discovery is an important aspect to working with data in formats such as JSON. Unlike relational databases, JSON data sets often do not have associated structural information. Consumers of such datasets are often left to browse through data in an attempt to observe commonalities in structure across documents to construct suitable code for data processing. However, this process is time-consuming and error-prone. Existing distributed approaches to mining schemas present a significant usability advantage as they provide useful metadata for large data sources. However, depending on the data source, ad hoc queries for estimating other properties to help with crafting an efficient data pipeline can be expensive. We propose JSONoid, a distributed schema discovery process augmented with additional metadata in the form of monoid data structures that are easily maintainable in a distributed setting. JSONoid subsumes several existing approaches to distributed schema discovery with similar performance. Our approach also adds significant useful additional information about data values to discovered schemas with linear scalability.

翻译：模式发现是处理JSON等格式数据的重要环节。与关系数据库不同，JSON数据集通常缺乏关联的结构信息。此类数据集的使用者往往需要遍历数据，试图观察文档间结构的共性以构建合适的数据处理代码。然而，这一过程既耗时又易出错。现有分布式模式挖掘方法具有显著的易用性优势，能为大规模数据源提供有用的元数据。但根据数据源的不同，为估算其他属性（以帮助构建高效数据管道）而执行的即席查询可能代价高昂。我们提出JSONoid——一种分布式模式发现流程，通过附加以幺半群数据结构形式体现的元数据来增强其功能，这种数据结构在分布式环境下易于维护。JSONoid的性能与现有多种分布式模式发现方法相当，同时还能以线性可扩展的方式为发现的模式附加大量有价值的数据值信息。