Common data types like dates, addresses, phone numbers and tables can have multiple textual representations, and many heavily-used languages, such as SQL, come in several dialects. These variations can cause data to be misinterpreted, leading to silent data corruption, failure of data processing systems, or even security vulnerabilities. Saggitarius is a new language and system designed to help programmers reason about the format of data, by describing grammatical domains -- that is, sets of context-free grammars that describe the many possible representations of a datatype. We describe the design of Saggitarius via example and provide a relational semantics. We show how Saggitarius may be used to analyze a data set: given example data, it uses an algorithm based on semi-ring parsing and MaxSAT to infer which grammar in a given domain best matches that data. We evaluate the effectiveness of the algorithm on a benchmark suite of 110 example problems, and we demonstrate that our system typically returns a satisfying grammar within a few seconds with only a small number of examples. We also delve deeper into a more extensive case study on using Saggitarius for CSV dialect detection. Despite being general-purpose, we find that Saggitarius offers comparable results to hand-tuned, specialized tools; in the case of CSV, it infers grammars for 84% of benchmarks within 60 seconds, and has comparable accuracy to custom-built dialect detection tools.
翻译:摘要:常见数据类型(如日期、地址、电话号码和表格)可能存在多种文本表示形式,而许多广泛使用的语言(如SQL)则存在多种方言。这些差异可能导致数据被错误解释,进而引发静默数据损坏、数据处理系统故障甚至安全漏洞。Saggitarius是一种新的语言和系统,旨在通过描述语法领域(即描述一个数据类型多种可能表示的上下文无关文法集合)来帮助程序员推理数据格式。我们通过示例介绍Saggitarius的设计,并提供其关系语义。我们展示了如何使用Saggitarius分析数据集:给定示例数据,它基于半环解析和MaxSAT的算法推断给定领域中与数据最匹配的文法。我们通过在包含110个示例问题的基准测试集上评估该算法的有效性,证明系统通常能在数秒内仅凭少量示例返回令人满意的文法。此外,我们深入开展了关于使用Saggitarius进行CSV方言检测的扩展案例研究。尽管具有通用性,但结果表明Saggitarius的性能与手动调优的专业工具相当;在CSV场景中,它能在60秒内为84%的基准测试推断出文法,且准确度与定制化的方言检测工具相近。